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No  hard  boundaries  between  categories.  The  most  dangerous 
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Figure  1*  Non-Traditional  Quadrants 


Background 

he  current  National  Defense  Strate¬ 
gy  was  built  upon  the  efforts  of  the 
2001  Quadrennial  Defense  Review 
(QDR)  and  the  dramatic  changes  in  the 
security  environment  and  lessons  learned 
in  the  post  9-1 1  world.  That  strategy  iden¬ 
tified  four  persistent  and  emerging  chal¬ 
lenges  in  this  new,  more  uncertain  area. 
The  four  challenges,  traditional,  irregular, 
catastrophic  and  disruptive,  are  depicted  in 
Figure  1 .  The  majority  of  modeling  activ¬ 
ity  since  the  Cold  War  ended  has  continued 
to  center  on  the  analysis  that  supports  tra¬ 
ditional  warfighting  as  we  moved  from  the 
two  simultaneous  Major  Theater  Warfare 
strategies  of  the  1 990s  to  the  sequential 
Major  Combat  Operations  strategy  of  this 
decade.  As  we  attempt  to  seek  ways  to 
develop  techniques  to  analyze  the  non -tra¬ 
ditional  areas,  we  have  found  that  our 
familiar  physics-bascd  existing  suite  of 
tools  are  not  well  suited  to  examine  the 
capabilities  of  the  Joint  Force  against  these 
new  challenges. 

During  the  73rd  MORS  Symposium  at 
the  US  Military'  Academy  in  West  Point, 
NY,  Mr  Trip  Barber,  OPNAV  NS  IB  and 
Dr  Jacqueline  Henningscn.  FS,  USAFA9 
(formerly  AFSAA),  discussed  the  need  for 
a  MORS  workshop  to  facilitate  communi¬ 
ty  identification  and  sharing  of  tools,  new 
techniques  and  methodologies  and  new 
developments.  They  wanted  the  workshop 
to  provide  immediate  benefit  to  modeling 
developments  and  analytic  support  to  pro¬ 
vide  insights  to  answer  the  post  2005  QDR 
questions  within  the  Department  and  to 


support  future  analysis  to  support  program¬ 
matic  and  acquisition  decision  making. 
Subsequently,  Mr  Barber  and  Dr  Hen- 
ningsen  agreed  to  co-sponsor  the  workshop. 

Workshop  Overview 

Workshop  preparations  began  in  earnest 
in  September  2005  with  the  approval  of  the 
Terms  of  Reference  (TOR)  by  Lhe  MORS 
Executive  Counci!  and  the  identification  of 
working  group  (WG)  chairs.  The  workshop 
objectives  as  defined  in  the  TOR  arc  shown 


in  Figure  2.  Working  group  chairs  were 
selected  early  on  and  tasked  writh  accom¬ 
plishing  a  significant  amount  of  pre-work¬ 
shop  preparation.  This  included  identifica¬ 
tion  of  key  stakeholders  relating  to  each  of 
the  working  groups,  identifying  their  non- 
traditional  analysis  gaps  and  deficiencies, 
and  populating  working  groups  with  the 
right  match  of  stake  holders  with  a  cross 
section  of  area/topic  experts  and  modelers 
to  encourage  innovation  and  out-of-the  box 

(See  MORS  WORKSHOP,  pr  32) 
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Workshop  Objectives 

*  Illuminate  user  Non-traditional  modeling  and  simulation 
gaps  and  deficiencies  in  the  context  of  the  defense 
security  challenges  and  emerging  changes  in  the 
defense  strategy 

*  Identify  areas  of  user  commonality 

*  Capitalize  on  Joint/Service/Agency  toolsets  and 
ongoing  model  development  for  the  larger  community 

-  Develop  partnerships  to  share  future  M&S 


Figure  2,  Workshop  Objectives 


MORS  WORKSHOP 

(continued from  p,  l) 

thinking.  Working  group  procedures  were 
issued  four  months  prior  to  the  workshop  to 
serve  as  a  guideline  and  aid  in  planning  for 
the  working  group  chairs.  Organizing  com¬ 
mittee  meetings  were  held  every  2-4  weeks 
over  the  five-month  period  preceding  the 
workshop.  The  Sponsors  were  updated  with 
group  plans  and  status  in  a  face-to-face 
meeting  with  local  area  chairs  approximate¬ 
ly  one  month  before  the  workshop.  A  sec¬ 
ond  meeting  was  held  the  week  prior  to  the 
workshop  where  fmal  plans  were  briefed  to 
the  Sponsors.  The  success  of  the  workshop 
was  directly  attributable  to  these  pre- work¬ 
shop  accomplishments. 

The  2006  QDR  report  and  a  draft  of  the 
new  Strategic  Planning  Guidance  (SPG) 
were  released  approximately  one  month 
prior  to  the  workshop.  As  a  result,  each 
working  group  was  asked  to  review  the 
QDR  report  and  draft  SPG  and  refocus  their 
efforts  to  be  consistent  with  the  new  guid¬ 
ance. 

The  first  half  day  of  the  workshop  fea¬ 
tured  a  plenary*  session  to  provide  guidance 
and  focus  for  the  workshop  breakout  ses¬ 
sions.  Col  Suzanne  Beers,  USAF,  MORS 
President,  welcomed  the  participants.  Col 
Beers  was  followed  by  Mr  Greg  Mclchcr, 
JHU/APL,  who  provided  the  host  welcome. 
Dr  Jacqueline  Henningsen,  FS,  USAF  A9, 
provided  the  Sponsor's  welcome  and  intro¬ 
duced  the  keynote  speaker,  VADM  Evan 
Chanik.  Director  Force  Structure,  Resource 
and  Assessment  Directorate,  The  Joint  Staff 
(DJ-8),  VADM  Chanik’s  keynote.  Meeting 
the  Nation  s  Challenges  of  Tomorrow-  How 
the  Operations  Research  Analysts  Can  Help » 
provided  insights  to  shape  working  group 
discussions.  He  identified  five  challenges 
for  the  workshop  and  for  the  Defense  ana¬ 
lytic  community'  in  general: 

1 .  Continue  to  broaden  open  and  collabora¬ 
tive  environments  to  study  issues  con¬ 
cerning  the  Department  of  Defense; 

2.  Continue  the  work  started  in  past  work¬ 
shops  and  the  analytical  methodology 
working  group; 

3 .  1  dertti  fy  required  capabi  lilies  to  fi  ght  our 
wars  in  the  future; 

4.  Develop  analytical  methodologies  to 
analyze  non-traditional  warfare;  and, 

5.  Provide  a  list  of  gaps  in  tools  and  meth¬ 
ods  for  development  within  the  research 
community. 


Other  pi  enary  speakers  included  Dr  John 
Hanley,  Institute  for  Defense  Analyses,  who 
discussed  the  applicability  of  traditional 
operations  analysis  techniques  to  non-tradi¬ 
tional  challenges.  Dr  II  Ian  a  Kass,  Chief  of 
Staff  of  the  Air  Force  Cyberspace  Task 
Force,  who  described  the  role  of  Cyber  War¬ 
fare  as  both  an  irregular  and  a  disruptive 
technology  and  challenged  the  audience  to 
look  for  ways  to  analyze  this  aspect  of  the 
non-traditional  spectrum,  and  COL  Patrick 
Kelly,  Office  of  the  Secretary  of  Defense- 
Policy,  who  provided  a  snapshot  of  QDR  and 
emphasized  integrated,  augmented  and  com¬ 
plementary'  analysis  and  the  need  to  employ 
a  broad  array  of  methods  and  tools.  Service 
perspectives  were  provided  by  Dr  Jacque¬ 
line  Henningsen,  USAF  A9,  Mr  Trip  Barber, 
OPNAV  N81,  Dr  George  Akst,  USMC 
MCCDC,  and  Dr  David  Markowitz,  USA 
Center  for  Army  Analysts  who  provided  fur¬ 
ther  challenges  and  focus  for  the  audience. 

YVG  1:  Irregular  Warfare/G WOT 

WG  1  examined  Irregular  Warfare  and 
the  Global  War  On  Terror  (rW/GWOT)  with 
an  emphasis  on  those  elements  that  did  not 
appear  to  have  dearly  applicable  methods 
for  analysis,  WG  1  used  a  combination  of 
speakers,  sub- working  groups,  scenarios, 
and  other  facilitation  techniques  to  engage 
the  audience  in  an  attempt  to  "think  outside 
the  box,”  The  use  of  sub-working  groups 
helped  to  focus  on  specific  areas.  The  sub- 
working  groups  designed  and  used  scenar¬ 
ios  as  a  tool  to  examine  unique  modeling  and 
analytic  challenges  and  stimulate  discussion 


on  irregular  missions  The  WG  focused  on 
identifying  tools  and  techniques  that  would 
help  associate  actions  with  effects  and 
results  for  situations  that  arc  currently 
thought  of  as  "hard”  such  as  human  deci¬ 
sions,  the  media,  influence  operations,  or 
dealing  with  population  dynamics.  They 
approached  this  challenge  by  defining  the 
problem,  looking  at  existing  techniques,  and 
identifying  ncwr  approaches.  The  three  mam 
WG  plenary'  sessions  were  built  around  this 
concept.  Finally*  the  WG  organized  their 
output  in  a  process  format  starting  with  the 
question  followed  by  data,  analysis,  tools 
new-  ideas  and  recommendations. 

Getting  the  question  right  was  identified 
as  the  most  important  step  in  the  process  fol¬ 
lowed  by  data  collection.  While  the  WG 
found  there  is  potentially  a  dearth  of  data 
available,  they  concluded  that  collecting  the 
data  may  be  very  difficult.  Various  methods 
of  collecting  data  were  debated  including 
data  mining  and  polling.  Additionally,  the 
WG  noted  there  are  barriers,  such  as  politi¬ 
cal  constraints,  which  may  prevent  collec¬ 
tion  of  data  at  all. 

One  recurring  subject  of  debate  w  ithin 
the  workshop  was  the  role  of  scientific  or 
operations  research  principles  as  they  apply 
to  non-traditional  areas,  Much  DoD  analy¬ 
sis  and  modeling  up  to  this  point  has  drawn 
on  the  physical  sciences  for  insight  regard¬ 
ing  decisions  and  operations.  However 
w  hen  confronted  w  ith  JW/GWrQT,  a  purely 
physical  science  model  for  gaining  insight 
begins  to  break  down.  This  means  wc  may 
have  to  admit  that  net  everything  falls  into 
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the  category  of  things  that  are  amenable  to 
the  approaches  taken  by  the  physical  sci¬ 
ences.  Secondly,  we  may  need  to  manage 
risk  associated  with  making  decisions  about 
processes  that  are  not  amenable  to  “scientif¬ 
ic’"  methods.  Many  of  our  recommendations 
are  about  ways  to  help  decision  makers 
make  better  decisions  and  manage  risk  in  sit¬ 
uations  where  there  is  a  single  point  solution. 

In  doing  analyses  it  is  helpful  to  have  the 
right  tools.  The  WG  examined  a  variety'  of 
potential  tools  available  to  deal  with  these 
issues,  but  none  of  them  stood  out  as  a  “gold 
nugget."  The  WG  concluded  that  a  variety 
of  tools  was  needed,  each  with  a  potentially 
different  theoretical  or  methodological 
approach,  designed  to  provide  a  spectrum  of 
insights  to  the  decision  maker.  In  addition, 
the  WG  found  that  most  of  the  tools  they 
examined  were  specialized,  and  thus 
required  some  modification  in  order  to 
address  a  particular  problem.  Most  of  the 
models  also  tend  to  be  complex  and  require 
an  extensive  developer/subject  matter 
expertise  base  to  set  up  and  run. 

The  WG  examined  a  number  of  missions 
to  look  for  potential  tools  that  would  be  use¬ 
ful  in  examining  related  questions.  The  pro¬ 
posed  missions  ranged  from  distributed 
operations  to  small  unit  operations.  The  WG 
found  that  few'  are  well  defined;  some  are 
overlapping,  and  some  are  not  mutually 
exclusive.  Since  this  taxonomy  did  not  yield 
any  insights  into  the  complexities  of  the 
problem,  the  WG  reverted  to  a  more  tradi¬ 
tional  set  of  definitions  based  on  missions 


assigned  to  special  operations  forces.  The 
first  step  in  finding  potential  tools  was  to 
examine  the  existing  traditional  warfare  tool 
suite.  Not  surprisingly,  artful  use  of  tradi¬ 
tional  modeling  and  simulation  (M&S)  tools 
yields  some  very  limited  insights  into  IW 
operations  that  incorporate  direct  action.  As 
expected,  none  of  the  traditional  tools  exam¬ 
ines  all  aspects  of  the  IW  mission  space. 
Most  irregular  warfare  missions  involve 
political,  military,  economic,  sociological, 
informational,  and  infrastructure  (PMESII)) 
interactions  in  addition  to  military  and  as  a 
result  are  not  well  addressed  by  traditional 
M&S  tools  (see  Figure  3), 

The  WG  also  tried  matching  ‘'unconven¬ 
tional  tools*11  to  the  mission  areas  to  assess 
model  goodness.  However,  this  was  some¬ 
what  hampered  by  a  lack  of  prior  studies  and 
analysis  in  the  area  of  IW/G  WOT  At  pres¬ 
ent,  there  are  few'  analytical  agenda  scenar¬ 
ios  that  cover  the  depth  and  breadth  of  IW. 
However,  once  these  are  completed  and  the 
analytic  baselines  developed,  a  better  eval¬ 
uation  can  be  undertaken.  These  studies 
would  also  reinforce  a  best  of  breed  model 
development  practice  to  achieve  better  mod¬ 
els  to  address  issues  in  the  IW  arena. 

There  are  multiple  technologies  that  may 
be  useful  in  examining  I W  including,  but  not 
limited  to,  agent  based  modeling,  systems 
dynamics  methods,  and  quantitative  com¬ 
putational  social  sciences.  It  is  not  dear  if 
any  particular  technique  applied  to  I W  prob¬ 
lems  will  generate  useful  insights.  Hence, 
the  WG  concluded  a  best  of  breed  approach 
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■  Domain  is  large,  nebulous,  complex,  and  also  not  well  understood 
because  of  competing  theories  and  lack  of  empirical  data.  Consider: 

*  expanding  the  role  of  analytical  wargames  as  tools  that  support 
decision-making  to  scope  the  problem  and  identify  key  issues, 
and  potential  solutions 

*  the  use  of  commercial  gaming  environments. 

*  use  of  polling,  “expert"  interviews,  etc 
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Jrregufar/GWOT  problems  often  involve  the  “soft"  or  social  sciences, 
e.g.  human  behavior,  media,  influence,  populations,  which  poses  a 
data  problem  (difficult  to  collect,  messy,  may  be  controversial). 


Figure  3.  Irregular/GWOT  Considerations 


is  needed  to  develop  better  models,  espe¬ 
cially  at  the  campaign  level  of  analysis.  The 
WG  recommended  that  several  different 
approaches  should  be  pursued,  leveraging 
prior  research  efforts  where  applicable.  Var¬ 
ious  tools  could  be  tried  in  prototype  studies 
and  the  results  could  be  fed  back  into  the  tool 
development  process. 

While  little  “low'  hanging  fruit”  was 
found,  WGl  highlighted  several  key  points 
and  issues: 

*  Board  and  Computer  Carnes,  While 
games  have  been  used  by  the  military  for 
many  years  to  consider  various  issues,  the 
WG  felt  that  games  and,  broadly,  gaming 
techniques,  might  be  better  able  to  address 
some  of  the  issues  that  come  up  in  dealing 
with  populations  and  human  factors. 
Research  is  needed  to  help  determine  the 
applicability  of  various  gaming  tech¬ 
niques  to  the  IW/G  WOT  problems. 

•  Analysis  and  Assessment  of  Risk,  Chang¬ 
ing  the  way  analysis  is  consumed  may  be 
the  most  important  and  cheapest  way  to 
implement  insights  gained  from  this  WG. 
It  may  also  be  the  hardest  to  implement. 
It  would  require  that  consumers  of  analy¬ 
sis  develop  ways  to  deal  with  uncertainty 
in  the  inputs  and  results.  It  would  also 
require  that  the  analyses  be  conducted  in 
ways  that  balance  risk,  and  develop  mul¬ 
tiple  possible  experimental  approaches 
and  outcomes.  What  level  of  uncertainty 
is  too  much  in  the  results?  How  do  you 
make  decisions  using  outcome  landscapes 
instead  of  single  instance  solutions? 

*  Wargames  and  Simulations.  Expanding 
the  role  of  wargames,  especially  for 
domains  that  do  not  lend  themselves  to 
modeling  or  where  data  is  not  available  to 
support  the  model,  would  aid  decision 
making.  Wargames  and  simulations  can 
sometimes  be  used  to  help  frame  and 
focus  decisions  and  play  both  an  analyti¬ 
cal  as  well  as  a  socialization  role  within 
the  decision  making  process. 

•  Ensemble  of  Approaches,  A  range  of  the¬ 
ories,  models,  and  methodologies,  that 
yield  a  myriad  of  outcomes  at  different 
levels  of  fidelity  are  needed  when  address¬ 
ing  IW/G  WOT.  The  WG  was  briefed  on 
a  recent  DARPA  effort  that  involved  mul¬ 
tiple  competing  models  that  examined 
government  stability.  Examining  the 
“landscape”  of  model  outputs  provided 
greater  insights  than  could  have  been 
gained  by  using  an  individual  model.  This 

(See  MORS  WORKSHOP,  p  34) 
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WG  3:  Catastrophic 
Current  Analytic  and  M&S  Capabilities 

*  Current  capabilities 

-  Engineering  level:  robust  individual  models  DTRA  Suite.  CAPS 

-  Tactical  /  operational  level  EPICS,  JCATS,  IWMDT 

-  Campaign  level:  legacy  models  (ITEM),  but  these  are  not  tailored  for 
domesttc  catastrophic  events 

-  Training  models:  significant  lack  of  simulator/stimulators/facilities 

•  Applications/examples  -  many  stand  alone  tools:  multiple 
tools  often  required  to  assess  a  scenario 

-  Defense  Collaboration  Tool  Suite/InfoWbrkSpace  (DCTS/IWS) 

-  Hazard  Prediction  Assessment  Capability  (HPAC) 

-  Consequence  Assessment  Toolset  (CATS) 

-  integrated  Munitions  Effects  Assessment  (IMEA) 

-  Vulnerability  Assessment  and  Protection  Option  (VAPO) 

-  Traditional  campaign  anaiysis/mission  level  tools  re-purposed  to 
assess  catastrophic  scenarios 


Figure  4.  Catastrophic  Toolset 


MORS  WORKSHOP 

(continued from  p.  33 ) 

concept,  of  using  multiple,  different 
approaches  to  address  the  same  problem 
was  felt  to  be  more  appropriate  for  exam¬ 
ining  IW  problems  than  any  one.  individ¬ 
ual  model  or  approach. 

■  Verification  and  Validation  (V&  V).  Tra¬ 
ditional  methods  used  for  validation  may 
not  apply  to  the  issues  discussed  in  this 
WG,  It  may  be  important  to  examine  tra¬ 
ditional  assumptions  about  reproducibili¬ 
ty  and  traceability  that  are  required  for 
valid  operational  analysis  when  consider¬ 
ing  subjects  like  human  behavior,  popula¬ 
tion  dynamics,  and  other  “soft  questions.” 
Ultimately  the  question  of  “validation” 
comes  down  to  the  role  of  the  decision 
maker  and  the  degree  of  risk  they  are  will¬ 
ing  to  assume  in  making  a  decision  based 
on  information  that  is  not  reproducible  or 
“scientific”  in  the  physical  sciences  sense. 
In  the  WG  discussions,  it  became  appar¬ 
ent  that  many  of  the  questions  that  deci¬ 
sion  makers  ask  may  not  be  answerable 
through  traditional  OR  techniques. 

In  summary,  WG  1  concluded: 

■  Problem  definition  is  a  critical  path  and 
will  likely  be  a  lengthy,  iterative  process 
for  non-tradkional  areas. 

*  There  is  a  great  need  for  valid,  real -world 
data  to  support  modeling  and  analysis.  It 
may  be  useful  to  examine  the  various 
impediments  to  collecting  this  data  includ¬ 
ing  legal,  programmatic,  and  other  issues. 

*  Using  an  ensemble  of  models  may  yield 
the  “landscape”  of  outcomes  for  decision 
makers.  One  difficulty  in  this  approach  is 
how  to  reason  across  such  an  ensemble  of 
disparate  tools  and  methods  and  commu¬ 
nicate  the  right  outputs  to  the  decision 
maker. 

*  The  application  of  wargaming,  simula¬ 
tions  and  commercial  gaming  techniques 
may  be  useful  in  gaining  insights  in 
IW/GWOT 

WG  2:  Disruptive 

WG  2  was  initially  established  to  exam¬ 
ine  the  Disruptive  quadrant  but  this  WG  was 
later  merged  into  WG  !  and  WG  3. 

WG  3:  Catastrophic 

WG  3  made  extensive  use  of  pre-work- 
shop  surveys  to  identify'  gaps  and  shortfalls 
in  the  catastrophic  quadrant.  The  WG  began 
by  deliberating  “what  constitutes  a  cata¬ 


strophic  event”  and  in  doing  so  examined  the 
15  National  Planning  Scenarios  developed 
by  the  Homeland  Security  Council  and 
Department  of  Homeland  Security  (DHS). 
Several  of  these  scenarios  involve  signifi¬ 
cant  loss  of  life  and/or  significant  social, 
political,  economic,  or  psychological  effects. 
The  other  component  of  the  catastrophic 
quadrant,  “Countering  Weapons  of  Mass 
Destruction  (WMD),”  was  defined  by  the 
WG  to  include  events  involving  WMD  inter¬ 
diction,  counter-proliferation,  and  deter/ 
defeat  the  use  of  WMD  by  state  and  non¬ 
slate  actors. 

The  WG  concluded  that  the  majority  of 
the  current  M&S  tools  and  capabilities  that 
address  the  two  catastrophic  areas  are  at  the 
engineering  and  mission  areas  of  fidelity. 
There  are  a  few  tools  available  at  the  tacti¬ 
cal  level.  There  is  some  effort  being  applied 
to  use  some  of  our  legacy  campaign  models 
(such  as  ITEM)  in  the  Homeland  Security 
(HLS)/Homeland  Defense  (HLD)  arena,  but 
generally  these  tools  are  not  tailored  for 
analysis  of  domestic  catastrophic  events. 

The  applications/examples  shown  in  Fig¬ 
ure  4  are  just  the  tip  of  the  iceberg  with 
regard  to  our  ability  to  model  WMD  effects. 
These  tools  are  generally  very  good  at  mod¬ 
eling  first  order  effects  and  the  spread/trans¬ 
mission  of  those  effects  from  the  source 
through  the  “target”  area.  Generally,  these 
tools  are  stand-alone  and  not  interactive 


models.  The  Defense  Threat  Reduction 
Agency  (DTRA)  is  currently  involved  in 
enhancing  several  of  the  M&S  tools  lo  bet¬ 
ter  support  their  w  arfighter  customer.  These 
tools  will  become  part  of  the  Integrated 
WMD  Toolset  and  will  be  available  through 
a  reachback  function  centered  on  the  DTRA 
Collaboration  Center. 

Key  analysis  gaps  and  deficiencies  and 
stakeholder  commonalities  were  drawn  from 
the  pre-workshop  surveys  and  further 
refined  during  the  workshop.  These  are 
depicted  in  Figure  5  The  WG  acknowl¬ 
edged  analyses  work  needs  to  be  coordinat¬ 
ed  or  collaborated  across  all  stakeholders, 
since  the  plans/CONOPS  for  each  stake¬ 
holder  will  be  influenced  or  dependent  on 
that  of  the  other  stakeholders.  These  stake¬ 
holder  interdependencies  must  be  captured 
in  the  modeling/analysis  to  provide  accurate 
outputs. 

The  WG  felt  analytical  support  of/partic¬ 
ipation  in  exercises  and  wargames  appeared 
to  be  the  best  starting  point  for  eliciting  the 
data  that  will  clarify  stakeholder  roles  and 
capabilities  for  HLD;  The  WG  concluded 
that  breaking  down  barriers  between  analy¬ 
sis  communities  within  DoD  and  between 
agencies  would  be  facilitated  by  a  standard 
data  format  and  by  methods  for  navigating 
around  classified  data. 

The  WG  identified  primary  M&S  short¬ 
falls  and  gaps  in  the  areas  of  data.  CON  OPS, 
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and  methods/tools. 

In  addressing  data,  the  WG  saw  three 
major  deficiencies: 

1.  A  lack  of  data  concerning  the  capabilities 
of  the  local,  state,  other  federal,  Mon- 
Govemment  Organizations,  and  interna¬ 
tional  agencies.  It  is  extremely  difficult 
to  predict  what  capabilities  will  be 
requested  from  DoD  in  response  to  a  cat¬ 
astrophic  event  when  we  have  no  idea  of 
what  portions  of  the  response  activities 
can  be  handled  before  DoD  is  requested 
to  “enter  the  fight” 

2.  The  lack  of  tools  in  the  biological  area. 
This  is  especially  true  concerning  some 
of  the  newer/engineered  bio  threats  and 
with  the  retransmission  or  secondary 
infections  from  the  agent. 

3.  The  lack  of  current  empirical  test  data 
available  to  test  our  models/tools  against. 
For  example,  in  the  nuclear  arena,  the  last 
open  air  nuclear  test  was  conducted  in 
1962  and  the  test  data  collected  in  that 
and  previous  tests  did  not  have  the  fideli¬ 
ty  that  we  would  want  to  collect  today. 

In  the  area  of  CQNOPS,  while  there  has 
been  some  effort  to  determine  what  capabil¬ 
ities/  tasks  are  required  in  response  to  a  cat¬ 
astrophic  event,  the  WG  ascertained  there 
have  been  no  real  efforts  to  determine  how 
these  responses  need  to  be  time-phased.  The 


majority  of  the  tools  that  support  the  cata¬ 
strophic  area  are  effects  related  and  are 
therefore  related  to  consequence  manage¬ 
ment.  The  WG  found  a  distinct  lack  of  effort 
concerning  analysis  of  pre-event  capabili¬ 
ties.  This  may  be  related  to  the  lack  of 
appropriate  metrics  in  this  area.  For  exam¬ 
ple,  how  do  you  measure  success  with  regard 
to  preventing  or  deterring  a  WMD  event? 

Regarding  methods  and  tools,  the  WG 
identified  the  following  concerns:  1)  the 
results  from  one  model  are  not  directly 
usable  as  inputs  to  another  model;  2)  there  is 
a  lack  of  tools  to  do  biological  event  model¬ 
ing;  3)  there  are  large  gaps  in  our  tools  to 
examine  the  socio-economical,  psychologi¬ 
cal,  and  political  effects  of  a  catastrophic 
event;  and,  4)  there  is  no  aggregated  tool  to 
examine  effects  at  the  campaign  level. 

The  WG  concluded  there  are  no  simple 
“tool”  solutions  to  the  analysis  of  cata¬ 
strophic  scenarios.  The  community  needs 
new  tool  /mo  deling/an  a  lysis  ideas  and  con¬ 
cepts,  Partnerships  between  the  various 
stakeholders  also  need  to  be  developed.  WG 
3  separately  reached  the  same  conclusions 
that  were  developed  in  WG  1  —  first  and 
most  important  is  defining  the  problem  cor¬ 
rectly  and  then  identifying  and  gathering 
data  before  developing  new  methodologies 
and  tools. 

In  summary,  the  WG  noted  that  there  is 
generally  a  lack  of  models/tools  above  the 


engineering  or  systems  level  of  fidelity. 
Before  development  of  higher  level,  more 
aggregated  models  can  proceed,  the  focus 
should  be  on  developing  good  data  and 
sources  for  that  data.  This  may  be  facilitat¬ 
ed  by  the  establishment  of  a  robust  intera¬ 
gency  community  of  interest  in  the  cata¬ 
strophic  arena.  The  WG  also  saw  a  need  to 
divide  the  catastrophic  mission  area  into  pre- 
and  post-event  areas.  Current  efforts  seem 
to  be  focused  on  the  post -event  or  conse¬ 
quence  management  actions,  but  we  cannot 
afford  to  neglect  analysis  focused  on  pre¬ 
venting  or  deterring  catastrophic  events. 

WG  4:  Deterrence 

Using  the  Strategic  Deterrence  Joint 
Operating  Concept  (SD  JOC)  as  the  con¬ 
ceptual  basis,  WG  4  posited  three  questions 
to  help  frame  discussions  within  the  work¬ 
ing  group.  These  questions  address  three 
persistently  thorny  issues  concerning  deter¬ 
rence  analysis, 

1.  How  can  analysts  characterize  and 
measure  uncertainty  concerning  deter¬ 
rence?  Uncertainty  comes  in  a  variety  of 
forms,  including:  the  uncertain  nature  of 
the  problem,  intelligence  available  to 
inform  the  analysis,  completeness  of 
understanding  of  the  adversary- s  decision 
making  process,  our  ability  to  develop 
and  communicate  deterrent  actions,  the 
manner  in  which  deterrent  actions  are 
received,  unplanned  actions  that  have 
unintended  consequences,  and  the 
dynamic  nature  of  strategic  events.  For 
example,  how  do  we  account  for  learning 
by  unidentified  (i,e„  potential)  adver¬ 
saries? 

2.  How  can  deterrence  analysts  determine 
and  manage  second-  and  third-order 
effects  in  deterrence  analysis?  While  we 
have  methods  to  identify,  and  to  some 
extent  measure,  the  main  effects  (or 
intended  effects)  of  our  deterrent  actions 
on  adversary  decision  making,  we  have 
difficulties  in  identifying  and  accounting 
for  cascading  effects  that  come  in  a  vari¬ 
ety  of  forms.  Some  spillover  effects  are 
geographical  (e.g„  neighboring,  region¬ 
al,  global);  some  are  temporal  (e.g,,  short 
term  deterrence  may  contribute  to  longer- 
term  instability);  while  others  cross  dis¬ 
ciplines  (e.g,,  economic  actions  having  a 
diplomatic  effect). 

3 .  How  can  analysts  estimate  the  magnitude 
of an  adversary's  response  to  a  proposed 

(See  MORS  WORKSKHOP^p.  36) 


WG  3:  Catastrophic 
Analysis  Gaps  and  Deficiencies 

*  Key  stakeholders  and  decision-makers 

-  USG  (Inter-agency)  ♦DOD/Services/COCQMS  ♦Local  Govts 

*  Allies  ♦Coalmens  ♦Industry 

-  State/Territorial  Govts/WG  ♦  National/ private  Labs  ♦First  Responders 

*  Key  analysis  gaps  and  deficiencies 

”  Plrce^ed  Jack  of  objective  goals,  defined  responsibilities,  action/ re  sources 
reflecting  stated  priorities 

-  Need  for  improved  interagency  coo  pe  ration/co  I  fa  bo  ration  /data  sharing 

-  Insufficient  understanding  of  complex,  primary  effects  and  secondary  effects 
(political,  economic,  social,  psychological) 

“  UttJe  quantification  of  local,  state,  federal.  DoD,  and  inf  I  response  capabilities 

-  I mmature/non-existent  operating  procedures,  data,  tools  for  evolving  mission  and 
requirements 

~  Barriers  to  sharrng  data,  plans  (classification  issues,  terminology,  ontology) 

*  Commonalities  between  stakeholders 

-  Desire  for  comprehensive,  systematic,  interagency  analysis 

-  Desire  for  integrated,  tactical/operational  tools 

-  Need  to  plan  for  response  &  future  investments 

-  Mission  focus 


Figure  5.  Catastrophic  Gaps  and  Deficiencies 
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MORS  WORKSHOP 

( continued from  p.  35) 

friendly 1  deterren  t  a  aion  ?  I  n  m  o  st  case  s  , 
we  have  methods  to  determine  the  gen¬ 
eral  nature  of  an  adversary's  response  to 
a  specific  deterrent  action  {e.gM  favor¬ 
able,  unfavorable).  What  we  lack  are 
methods  to  determine  “how  favorable?*’ 
or  “how  unfavorable?”  that  will  ulti¬ 
mately  help  us  develop  coherent  pack¬ 
ages  of  deterrent  actions. 

As  an  initial  synthesis  of  the  groups7  dis¬ 
cussions,  Figure  6  depicts  die  multi -dimen¬ 
sional  nature  of  the  problem  contrasted  with 
some  of  the  more  promising  operations 
research  (OR)  methods  and  tools  available 
to  aid  in  deterrence  analysis.  The  group 
immediately  and  unanimously  saw  the  need 
for  a  suite  of  tools  for  even  the  most  basic 
of  deterrence  analysis  problems.  This  con¬ 
curs  not  only  with  the  other  working 
groups,  but  also  with  the  findings  from 
other  analytic  venues.  Underlying  this  suite 
of  methods  and  tools  is  the  need  for  data, 
and  the  nature  of  that  needed  data  is  chang¬ 
ing,  just  as  the  concept  has  changed.  The 
reminder  at  the  bottom  of  the  chart  serves 
to  bring  us  back  to  the  sometimes-over- 
looked  mandate  to  define  the  problem  cor¬ 
rectly  (i.e.t  getting  the  question  right)*  In 
this  case,  the  focus  of  a  substantial  part  of 
deterrence  analysis  is  to  help  develop  sets 
of  actions  that  will  serve  to  achieve  deter¬ 
rence  or  stability.  What  is  often  discussed, 
however,  are  the  difficulties  we  have  in 
quantifying  the  probability  of  an  adver¬ 
sary's  response.  While  interesting,  that 
problem  is  usually  not  the  effective  focus  of 
the  analysis  and  serves  to  distract  the  ana¬ 
lysts  and  decision  makers.  In  total,  this 
graphic  represents  an  initial  functional 
decomposition  of  the  problem  and  helped 
the  working  group  next  identify  gaps  and 
potential  solutions. 

Combining  the  functional  decomposi¬ 
tion  with  the  understanding  of  an  engineer¬ 
ing  design  process  yields  a  framework  to 
assess  method  and  tool  availability  for 
deterrence  analysis.  This  is  certainly  not  a 
definitive  mapping  of  tools  to  problems!  In 
fact,  even  if  we  could  account  for  all  sig¬ 
nificant  methods  and  tools,  the  mapping 
would  be  incomplete  since  it  potentially 
changes  based  on  the  nature  of  the  problem. 
This  means  the  problem  definition  step  in 
part  determines  wrhich  tools  are  appropriate 
fo r  a n al y s i s .  In  som e  cases ,  thes e  m ethod s 


WG  4:  “ Deterrence ” 


Attacking  the  Deterrence  Analysis  Problem 


Oimensions/f  actors 
-Players  (fed,  capabilities  &  limitations) 
“Decision  s/actions 
-Competition/coopa  ration 
-Deterministic/stochastic 
-Goal  sfobj*ctives  {a  e  v  mu  Hi  pi  e. 
Political,  econ,  mil?) 

-Payoffs  (+/-,  Costs.  Benefits)  reduce  prob 
of  achieve,  reduce 
benefit s/increasa  costs, 
preference  ordering 
-Stages/phases 

-Strategies  (risk  averse,  mini-max.  rrraxi- 
min) 

-  Inters  ctiont/lnfluences  {2nd,  3*  order 
effects) 

-Others . 


OR  Methodologies  {Suite) 

■Dynamic  Prog-amming  (stocri  astic  or 
oetarminjstic) 

-Game  Theory -{»  2  person ),  | non  0-  sum) 
I  multistage} 

-Experimental  Gaming 
-influence  Diagrams 
-Cost'Oeneftts  Analyses 
-Risk  Ar firsts  (p( event]  *  consegu&ote] 
-Decision  Trees  f  Decision  Analysis  (Value. 
Uifty) 

-Cognitive  Mapping 


Difference 

Today: 

-  Who 
*  Wiat 

-  How 

-  Context 


Application  of  Methodology  depends  on  the  dimensions  AND  the  question 
being  asked  or  the  decision  to  be  made! 


Appropriate?:  Do  certain  actions  contribute  (detract  from)  deterrence  or  stability? 
Bad?:  What  is  the  probability  that  action  X  will  deter  actor  Y  from  doing  Z? 


Figure  6*  The  Deterrence  Problem 


and  tools  have  already  been  applied  to  the 
indicated  analytic  tasks  and  tn  other  cases, 
they  have  not.  More  importantly,  the  group 
identified  a  number  of  tools  that  theoreti¬ 
cally  could  be  applied  to  certain  tasks,  but 
whose  data  collection  would  likely  drain  the 
national  budget,  or  are  otherwise  complete¬ 
ly  infeasible*  For  example,  mathematical 
network  models  used  to  examine  complex 
systems  might  aid  in  understanding  deci¬ 
sion  making  processes  in  terronst  networks, 
but  that  understanding  would  likely  require 
describing  each  node  and  edge  in  the  net¬ 
work  to  some  sufficient  level. 

While  the  method  and  tool  mapping 
highlights  some  general  potential,  it  does 
not  capture  specific  strengths  and  weak¬ 
nesses,  which  specific  tools  are  best  suited 
for  specific  problems,  or  what  priorities 
among  tools  and  gaps.  Instead  the  WG 
drew-  these  general  conclusions: 

*  As  expected,  there  is  no  “perfect  storm" 
that  will  address  all,  or  even  most,  of  the 
aspects  of  the  deterrence  analysis  prob¬ 
lems. 

*  We  face  significant  gaps  in  two  of  the 
three  focus  areas  laid  out  at  the  begin¬ 
ning:  determining  the  impacts  of  second 
and  third  order  effects,  and  determining 
the  magnitude  of  an  adversary’s  response. 
However,  the  focus  area  of  characteriz¬ 
ing,  reducing  and  managing  uncertainty 
may  well  be  substantially  covered  by  well 
developed  risk  analysis  methods. 


*  As  mentioned  previously,  the  data 
requirements  for  some  approaches  likely 
rule  out  the  practical  use  of  those 
approaches. 

*  Because  of  the  lack  of  data  problem,  the 
human-in-the-loop  (HITL)  approaches 
(e.g.,  role  playing,  wargaming)  are  often 
the  best  choice.  This  is  completely  rea¬ 
sonable  given  the  current  state  of  the  an 
for  deterrence  analysis,  HITL  methods 
are  often  excellent,  and  inexpensive, 
choices  for  addressing  formative  prob¬ 
lems* 

The  timing  of  the  workshop  was  oppor¬ 
tune  for  the  further  development  of  the 
deterrence  concept.  USSTRATCGM  is 
DoD’s  agent  for  maintaining  and  revising 
the  Strategic  Deterrence  Joint  Operating 
Concept  (SD  JOC).  In  February  2006,  that 
concept  w'as  mid-way  through  its  first  revi¬ 
sion  and  the  officer  from  USSTRATCGM 
responsible  for  that  revision  was  a  partici¬ 
pant  in  this  WG.  Accordingly,  the  insights 
gained  during  the  workshop  are  being 
incorporated  into  the  SD  JOC  revision, 

WG  5:  Decision  Making  and 
Integration 

initially  the  co-chairs  ofWG  5  struggled 
with  the  definition  of  “decision  making  and 
integration"  as  applied  to  this  workshop. 
Numerous  discussions  with  workshop 
sponsors  resulted  in  a  refined  charter  that 
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focused  on  assessing  the  implications  of 
QDR  2005,  Specifically,  the  group  focused 
on  assessing  the  ability  of  the  Department 
to  examine  the  capabilities  that  QDR  iden¬ 
tified  as  critical  for  meeting  the  strategic, 
operational  and  tactical  objectives  of  four 
focus  areas; 

1 .  Defeating  terror  networks; 

2.  Defending  the  homeland  in  depth; 

3.  Shaping  choices  of  countries  at  strategic 

crossroads;  and, 

4.  Preventing  the  acqui  sition  or  use  of 
WMD. 

In  order  to  provide  context  for  follow-on 
analysis,  WG  5  debated  the  future  environ¬ 
ment,  including  potential  changes  to  the 
threat,  strategic  drivers  and  advances  in 
technology.  Under  this  redefined  future, 
considerations  for  future  force  employment 
were  discussed,  and  then  mapped  to  the  set 
of  required  capabilities  listed  in  the  2005 
QDR  report. 

To  accomplish  this  task  the  WG  sub¬ 
divided  itself  into  four  subgroups  —  one  for 
each  of  the  four  QDR  focus  areas.  Each 
group  was  provided  the  same  task,  and 
asked  to  proceed  independently.  The  objec¬ 
tive  was  to  look  at  each  focus  area  and  inte¬ 
grate  the  results,  identifying  common  ele¬ 
ments  of  the  assessments  of  future  threats, 
the  strategic  environment  and  opportunities. 
The  groups  were  also  asked  to  identify  key 
decision  makers  —  and  postulate  the  type 
of  questions  they  might  ask  in  each  of  the 
four  focus  areas. 

In  describing  the  future  environment,  the 
sub-groups  concluded: 

*  The  proliferation  of  data  and  information, 
coupled  with  the  ability  to  rapidly  share 
that  information,  creates  a  level  playing 
field  for  the  US,  and  its  partners  and 
adversaries.  While  data  is  expected  to  be 
plentiful,  the  challenge  will  continue  to 
be  in  mining  information  to  find  elements 
that  contribute  to  problem  description, 
analysis  and  solution, 

*  Uncertainties  in  the  expected  degree  of 
government  agency  cooperation,  future 
levels  of  funding  and  the  evolution  of 
partnerships  (both  within  the  structure  of 
government  and  internationally)  present 
challenges  for  effective  defense  of  the 
homeland, 

*  The  future  environment  will  be  complex 
and  driven  by  numerous  factors.  Shap¬ 
ing  objectives  will  be  complicated  by 
interests  that  extend  beyond  traditional 
government  relationships,  and  wilt  be 


influenced  by  differing  levels  of  techno¬ 
logical  capability,  operational  capabilities 
and  system  interoperability  amongst 
cooperating  military  forces,  law  enforce¬ 
ment  and  other  agencies. 

•  Future  catastrophic  events  will  shape 
American  policy,  decisions  and  courses 
of  action. 

AN  four  subgroups  identified  opportuni¬ 
ties  for  (and  perhaps,  the  necessity  of) 
increased  cooperation,  collaboration  and 
integration  across  organizational  bound¬ 
aries,  The  groups  felt  that  the  nature  of  the 
non-traditional  security  challenge  will 
require  cooperation  and  integration;  not 
only  organizationally,  but  from  force  plan¬ 
ning,  resource  allocation  and  operational 
perspectives  as  well.  The  groups  saw  the 
first  step  for  military  operations  research 
analysts  as  increased  multidisciplinary  par¬ 
ticipation  in  studies  and  analyses  and  sur¬ 
mised  that  we  will  need  to  include  psychol¬ 
ogists,  sociologists,  economists,  inter¬ 
national  relations  experts,  and  many  other 
disciplines. 

The  WG  found  that  deficiencies  lie  in 
our  ability  to  measure  how  well  these  capa¬ 
bilities  answer  non-traditional  security  chal¬ 
lenges.  and  that  these  shortfalls  will  need  to 
be  mitigated  in  order  to  better  understand 
resource  allocation  requirements  -  not  only 
within  DoD,  but  across  all  government 
agencies  tasked  with  supporting  response  to 
non-traditional  security  challenges  we  will 


face  during  the  long  war.  As  depicted  in  Fig¬ 
ure  7  the  WG  presented  some  analysis  that 
indicates  that  the  nature  of  these  challenges 
may  call  for  a  broadening  of  our  analytic 
toolkit,  to  include  wider  use  of  social  sci¬ 
ence  and  risk  analysis  methodologies,  par¬ 
ticularly  in  the  areas  of  WMD  and  defense 
of  the  homeland. 

In  conclusion,  WG  5  agreed  that  the 
uncertain  and  potentially  dangerous  future 
requires  the  use  of  scenario- based  planning. 
The  scenarios  must  be  creative  enough  to 
enable  us  to  think  like  our  adversaries.  The 
ability  to  analyze  complex  and  adaptive 
adversaries  will  be  critical,  and  analysis 
augmented  by  “red  cell**  insights  and  the 
social  sciences  will  be  necessary'. 

Concluding  Remarks  from  the 
Synthesis  Group 

fn  assessing  the  common  themes  across 
all  working  groups,  the  Synthesis  Group 
focused  on  people,  data  and  V&V. 

People ,  A  common  denominator  across 
the  working  groups  was  the  recognition  that 
the  military  operations  research  communi¬ 
ty  (which  is  currently  largely  comprised  of 
mathematicians,  physicists,  ORs,  engineers 
and  scientists)  will  need  to  be  augmented  by 
other  disciplines  such  as  sociology,  anthro¬ 
pology,  regional  experts,  economy,  politi¬ 
cal  science,  and  psychology.  It  will  not  be 

f See  MORS  WORKSKHOPvp  38) 


Potential  Resource  Allocation  Methods' 

-  New  focus  on  risk  analysis  methods 


Caveat  -  WG  5  did  not  examine  specific  moaeis  arm  simulations 


Next  steps 

-  For  each  QDR  focus  area  and  across  the  focus 
areas 

*  Use  creative  new  scenarios 

*  Define  functions 

*  I  d  ent  ify  ca  p  a  b  if  ities  to  p  erf  or  m  fun  ction  s 
■  Develop  measures  for  each  capability 

*  Develop  models  and  simulations  to  calculate  measures 


Figure  7,  Analysis  Methods  for  QDR  Focus  Areas 


PHALANX 


7 


June  2006 


MORS  WORKSHOP 

(continued from  p  3  7) 


Methods  and  Techniques 


*  Seminar  wargaming 

*  Human-in-the-loop  simulations 

*  Commercial  gaming  environments 

*  Graph  theory  and  network  analysis 

*  Influence  and  Bayesian  network  tools 

*  Game  theory 
-  Risk  analysis 

*  Value  focused  thinking 

*  Analytical  hierarchy  process 

*  Interviews  and  Polling 

*  Lessons  Learned  Data 

*  Data  Mining 

*  Historical  Studies 


Figure  8.  Potential  NTM  Analytic  Methods  and  Techniques 


sufficient  to  leave  these  disciplines  to  aca¬ 
demia  and  reach  back  to  sufficiently 
address  non-traditional  areas.  We  need 
multi-disciplinary  learns  within  the  Depart- 
ment  to  work  collaborative!)'  on  the  ana¬ 
lytical  issues  for  non-traditional  warfare. 

Data.  Physics-based  data  is  well  under¬ 
stood  by  the  MORS  community.  The  soft 
sciences  are,  however,  a  fairly  new  domain. 
The  design  of  experiments  will  also  be  very 
important.  Methods  for  collecting  data  will 
have  to  be  improved.  Many  data  sets  will 
be  noisy  or  sparse  and  different  techniques 
for  analysis  will  be  required.  Data  mining 
technologies  will  need  to  be  incorporated. 
Sources  that  are  outside  of  the  norm  will  be 
required  such  as  mining  newspapers, 
industry  reports,  global /regional  economic 
data,  web  postings,  political  satire,  aca¬ 
demic  publications,  etc.  Defining  how  data 
feeds  models  and  methods  will  likely  be 
more  challenging. 

V&  V  V&V  was  a  common  theme  in 
working  group  discussions.  There  is  clear¬ 
ly  a  need  for  reasonable  and  sufficient 
V&V,  while  realizing  the  difficulty  of 
accomplishing  V&V  when  there  is  little 
real  world  history  or  test  results  to  form  a 
basis  of  comparison.  Also,  there  is  a  need 
to  accomplish  V&V  of  the  analytic  meth¬ 
ods  and  the  supporting  data  to  build  the 
confidence  of  decisions  makers  when  pre¬ 
sented  w'ith  analysis  using  new'  methods 
and  tools  and  relying  on  data  from  unfa¬ 
miliar  data  sources. 

In  conclusion,  the  Synthesis  WG 
assessed  the  workshop  accomplishment  in 
terms  of  achieving  its  objectives  of  review¬ 
ing  current  data  processes,  identifying  best 
practices  and  standards,  identifying  and 
assessing  potential  sources  of  data,  and 
educating  the  community  on  the  new  data 
directive.  The  WG  concluded  the  workshop 
made  excellent  progress  on  two  of  those 
objectives  (i.e.,  review  current  data 
processes;  and  educate  the  community'  on 
the  new  data  directive),  and  good  progress 
against  the  remaining  two  objectives  {i,e„ 
identify  best  practices  and  standards;  and 
identify  and  assess  potential  sources  of 
data).  Given  the  highly  complex,  multi¬ 
dimensional  problem  of  interest,  these  are 
major  accomplishments. 


Overarching  Recommendations 
and  Way  Ahead 

Overarching  Recommendations.  While 
there  is  some  new  tool  development  ongo¬ 
ing,  the  workshop  concluded  that  there  are 
no  “revolutionary”  new  tools.  Rather,  wc 
are  still  largely  engaged  in  defining  the 
problem,  identifying  and  gathering  data, 
and  investigating  many  different  method¬ 
ologies  for  analysis.  Some  emerging  tools 
were  identified  which  are  worth  investigat¬ 
ing  for  the  purposes  of  exploring  compet¬ 
ing  theories,  examining  consequences  of 
explicit  or  tacit  assumptions  and  providing 
structured  framework  for  ideas.  However, 
in  taking  a  holistic  approach  to  this  type  of 
analysis,  a  w  ide  range  of  techniques  should 
be  considered.  Some  examples  are  depict¬ 
ed  in  Figure  S. 

To  the  extent  possible,  stakeholders 
should  leverage  other  ongoing  efforts,  such 
as  DTRA’s  Modeling  Suite  for  CBRNE  and 
O SD 1  s  Irrcgu  I  ar  Warfare  Suite.  In  add i ti on 
to  participation  in  MORS  events,  we  also 
need  to  investigate  other  forums  to  share 
ideas  and  approaches  for  analysis  of  non- 
traditional  areas. 

Wa$  Ahead  This  workshop  constituted 
a  “step  along  the  wav,"  not  an  end.  It  should 
be  considered  just  one  in  a  sequence  of 
workshops  that  will  help  the  defense  analy¬ 
sis  community  develop  new  methodologies, 
methods,  and  tools  to  address  the  non-tra¬ 
ditional  challenges.  This  includes  develop¬ 
ing  m  e  t  h  odo logics  that  faci  l  i  late  asses  si  ng 


and  balancing  risk  and  the  appropriate 
V&V  for  those  methodologies  that  will 
enable  decision  makers  to  quickly  develop 
an  understanding  of  the  resulting  analysis 
and  confidence  in  those  results, 

Finally,  as  we  introduce  new  methods 
and  tools  it  will  take  time  for  the  decision 
makers  we  support  to  develop  the  level  of 
understanding  and  comfort  they  have  in 
studies  using  the  methods,  tools  and  mod¬ 
els  they  are  familiar  with  from  years  of  see¬ 
ing  the  results  of  studies  using  the  MCO- 
based  scenarios.  All  of  the  working  groups 
recognized  the  fact  that  we  need  to  first 
understand  the  types  of  problems  we  are 
attempting  to  solve  and  the  need  to  step 
through  a  1  st  order  solution  process.  Mod¬ 
els  may  not  be  required  to  solve  the  prob¬ 
lem,  but  ifa  model  is  required  oruseful, rec¬ 
ognize  the  need  for  having  an  ensemble  of 
available  models  that  we  can  use  when  try  ¬ 
ing  to  solve  the  problem  at  hand.  Most 
importantly,  DoD  must  ensure  that  it 
remains  a  proactive  participant  along  w'ith 
the  other  communities  identified,  in  the 
problem  solving  process. 
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Analysis  for  Non-Traditional  Security 
Challenges:  Methods  &  Tools 


Military  Operations  Research  Society 
(MORS)  Workshop 

WG  1  -  GWOT/Irregular 
Outbrief 


This  is  the  out  brief  from  Working  Group  1.  We  examined  GWOT  and  Irregular 
Warfare  with  an  emphasis  on  those  elements  that  we  did  not  feel  had  clearly 
applicable  methods  for  analysis.  The  working  group  was  composed  of  80 
registered  individuals  of  which  55-60  attended  on  a  regular  basis  and  was  divided 
for  part  of  the  time  into  ten  sub- working  groups.  At  least  60%  of  the  individuals 
identified  themselves  as  operations  research  analysts;  all  of  the  participants  are 
responsible  for  supporting  their  Service,  COCOM,  or  agency  with  analysis  on 
irregular  warfare  problems.  Individuals  were  present  from  the  Joint  Staff,  the 
Services,  SOCOM,  CIA,  academia,  industry,  FFRDCs,  UK,  and  Canada.  The 
sub- working  groups  conducted  a  scenario-based  analysis  of  our  modeling  and 
analysis  capabilities  for  GWOT  and  Irregular  Warfare  related  areas.  The  sub¬ 
working  groups  used  the  scenarios  as  a  tool  with  which  to  focus  on  specific 
modeling  and  analysis  challenges. 

This  briefing  is  structured  as  follows:  1)  a  briefing  on  the  overall  results  from  the 
working  group  plenary  sessions;  and,  2)  show  two  individual  slides,  and  two  full 
briefings,  from  different  sub-working  groups.  Because  there  were  ten  total  sub¬ 
working  groups  it  was  not  possible  to  show  the  outcome  of  all  sub- working 
groups. 
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WG  1  -  GWOT/Irregular 


fil635 

MHttw  DfWVtKma  >m»frt  SacUn 


•  Complex  topic 

•  Focused  on 

♦  Action  ->  Effect  ->  Results 

♦  Hard  problems  involving 

Human  factors,  media,  influence  and  populations 
+  DIME/PMESII 

■*  Analysis  in  support  of  Headquarters,  Combatants,  not  training 
etc. 

♦  Understanding  the  problem  and  existing  techniques 

♦  Identifying  new  approaches  that  would  work 

•  Structured  results  as  process 


We  felt  a  complex  topic  required  an  innovative  approach  toward  this  working 
group.  Thus  we  used  a  combination  of  speakers,  sub- working  groups,  scenarios, 
and  other  facilitation  techniques  to  engage  the  audience  in  an  attempt  to  think 
outside  the  MORS  box,  We  believe  we  accomplished  that  goal. 

The  working  group  focused  on  what  we  considered  to  be  the  overall  charter  from 
the  Sponsors:  to  identify  tools  and  techniques  that  would  help  them  associate 
actions  with  effects  and  results  for  situations  that  are  currently  thought  of  as  hard. 
Examples  of  hard  problems  include  anything  having  to  do  with  human  decisions, 
the  media,  influence  operations,  or  dealing  with  population  dynamics.  We 
approached  this  challenge  by  trying  to  understand  the  problem,  look  at  existing 
techniques,  and  identify  new  approaches.  The  three  main  plenary  sessions  were 
built  around  this  concept. 

The  rest  of  the  main  portion  of  this  briefing  is  structured  around  a  concept  of  the 
analytical  process,  starting  with  the  question  and  ending  with  the  tools  needed  to 
conduct  analysis.  The  completion  of  this  working  group  brief  will  discuss  new 
concepts  and  WG  recommendations. 
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WG  1  -  It’s  a  Process 

Hlktsry  DfMntkiec  fliiWth  Sofhfii 

•  The  question 

•  The  data 

•  The  analysis 

•  The  tools 

•  New  ideas 

•  Recommendations 

This  is  the  process  for  the  WG  1  brief,  essentially  an  outline  of  both  the  briefing 
and  any  analytical  process.  After  this  section  of  the  brief  there  will  be  briefings 
from  the  sub-working  groups. 
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WG  1  -  Get  the  Question  Right 


IH&S5 


•  Goals  and  questions  are  often  unclear 

♦  We  may  lack  basic  frameworks  for  understanding  the  problem 

■*  Use  frameworks  where  they  exist 

•  You  can  model  anything,  but  it  may  ultimately  not  be 
very  helpful 

•  The  question  is  important 

♦  Because  soft  issue  models  often  need  to  be  customized  for  the 
question 

♦  And  data  may  not  exist,  or  may  be  complicated  and  difficult  to 
collect 


The  most  important  thing  you  cart  do  to  ensure  an  analysis  (or  model,  or  game) 
will  be  successful  is  to  make  sure  you  are  answering  the  right  question. 
Unfortunately  in  this  area  we  may  lack  basic  frameworks  that  allow  us  to 
understand  enough  to  ask  the  right  questions.  During  the  final  plenary  session  we 
had  a  discussion  of  frameworks  for  the  interagency  aspects  of  irregular  warfare, 
something  that  is  both  challenging  and  necessary  if  we  are  to  be  able  to 
understand  enough  to  build  appropriate  analyses. 

Unfortunately,  even  if  you  don’t  ask  the  right  question,  it  is  possible  for  analysts 
and  modelers  to  go  off  and  start  working  anyway.  Usually  what  happens  is  that 
the  sponsor  realizes  that  the  analysis  is  heading  in  the  wrong  direction  and 
provides  course  correction.  Unfortunately  getting  the  question  wrong  may  be 
more  dangerous  in  this  case.  In  our  plenary  sessions  we  found  many  of  the 
models  we  were  dealing  with  needed  to  be  modified  or  customized  for  each 
question.  Likewise  data  was  non-existent  and  difficult  to  obtain.  If  models  have 
already  been  modified,  or  the  wrong  data  obtained,  it  may  be  more  difficult  in 
this  case  to  change  the  question  than  it  is  during  a  conventional  analyses. 
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WG  1  -  We  Need  Data 


li1©H5 

HtWtMty  UmiHrH  S«Mr 


•  Lack  of  data  poses  a  significant  problem 

•  Discussed  polling  as  an  example 

♦  Legal  constraints 

♦  Political  constraints 

♦  Methodological  constraints 

•  Much  of  the  data  we  need  involves  “human”  data  that 
is  difficult,  messy,  and  controversial  to  collect 

•  Simulations  can  only  go  so  far  when  compared  to  real 
world  data 


During  the  course  of  our  discussions  there  were  several  recurring  themes.  One 
was  that  there  is  a  dearth  of  data  that  can  support  our  analysis,  or  help  us  with  our 
models.  Polling  was  discussed  as  one  example  of  a  data  source  that  would  be 
helpful  (if  done  correctly)  but  is  difficult  to  obtain.  There  appear  to  be  various 
legal,  political,  and  methodological  constraints  to  conducting  polls  for  purposes 
of  analysis. 

In  general  this  points  out  that,  for  the  type  of  problems  we  are  considering  here, 
collecting  the  data  may  be  difficult.  As  people  are  involved  there  may  be  human 
subject  research  protocols  that  need  to  be  followed.  As  the  people  may  be  from 
sensitive  populations,  political  considerations  may  prevent  much  data  from  being 
collected.  This  limitation  is  one  that  may  need  to  be  considered  if  the  goal  is  to 
improve  our  ability  to  analyze  these  types  of  problems. 

Unfortunately  simulations  can  only  go  so  far  when  compared  to  real-world  data. 
Analysis  based  on  data,  even  fuzzy,  messy,  data  is  much  more  likely  to  provide 
decision  maker  insight  than  simulation  results  alone. 
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WG  1  -  The  Analysis 


lilGH5 

MMbrr  Qixnoam  Imki.  Soc^T, 


•  Operations  research  is  the  application  of  scientific 
methods  to  operational  problems 

♦  We  may  have  to  admit  that  some  things  do  not  fall  into  the  category 
of  physical  science 

♦  This  means  managing  the  risk  associated  with  decisions  in  new 
ways 

•  We  may  want  to  explore 

♦  New  decision  making  models  and  processes 

♦  Tools  that  may  not  be  capable  of  being  validated 

♦  New  techniques,  such  as  multiple  instances  of  models  and  theories 


Once  the  data  is  collected  and  the  questions  asked,  its  time  to  do  the  analysis. 

One  recurring  subject  of  debate  within  the  conference  was  the  role  of  scientific  or 
operations  research  principles  as  they  apply  to  the  types  of  problems  we  are 
discussing  here.  Much  DpD  analysis  and  modeling  up  to  this  point  has  drawn  on, 
the  physical  sciences  for  insight  regarding  decisions  and  operations.  However 
when  confronted  with  GWOT  or  Irregular  Warfare  a  purely  physical  science 
model  for  gaining  insight  begins  to  break  down. 

What  does  this  mean? 

Well,  first  we  may  have  to  admit  that  not  everything  falls  into  the  category  of 
things  that  are  amenable  to  the  approaches  taken  by  the  physical  sciences. 

Second,  if  we  accept  the  first,  then  we  may  need  to  manage  risk  associated  with 
making  decisions  about  things  that  are  not  amenable  to  scientific  methods.  Much 
of  our  recommendations  are  about  ways  to  help  decision  makers  make  better 
decisions  and  manage  risk  in  situations  where  there  is  no  one  right  answer. 
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WG  1  -  The  Tools 

mens 

HUttvy  OfMtKH  tanh  SocMy 

•  No  new  or  revolutionary  tools 

•  Considerable  skepticism  about  any  one  model’s  ability 

to  solve  a  problem 

•  All  the  existing  tools  tend  to  be 

♦  Specialized 

♦  Expert  intensive 

In  doing  analyses  it  is  helpful  to  have  tools.  Much  of  our  working  group’s 
discussion  was  about  tools,  do  they  exist,  what  can  they  do,  is  there  anything 
really  important  that  we  are  missing? 

The  answer  was  not  much.  While  there  are  tools  available  to  deal  with  these 
issues,  none  of  them  stood  out  as  the  “golden  BB”  that  we  were  looking  for.  In 
fact  there  was  great  skepticism  about  any  one  model’s  ability  to  address  a 
particular  problem.  As  shown  later  the  WG  concluded  that  what  w-as  likely 
needed  was  a  variety  of  tools,  each  with  a  potentially  different  theoretical  or 
methodological  approach,  designed  to  provide  a  spectrum  of  answers  to  the 
decision  maker. 

In  addition  we  found  that  most  of  the  tools  we  considered  were  specialized,  and 
thus  required  some  modification  in  order  to  address  a  particular  problem.  The 
models  also  tended  to  be  “expert  intensive”  in  terms  of  needing  execution  support 
from  those  who  built,  or  at  least  were  well  trained  in,  the  model. 
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WG  1  -  Existing  Models 
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The  WG  was  tasked  to  examine  a  number  of  missions  and  find  potential  tools 
that  would  be  useful  in  examining  related  questions.  The  proposed  missions 
ranged  from  distributed  operations  to  small  unit  operations.  Few  are  well 
defined;  some  are  overlapping,  and  some  are  not  mutually  exclusive.  Since  this 
taxonomy  did  not  yield  any  insights  into  the  complexities  of  the  problem,  the  WG 
reverted  to  a  more  traditional  set  of  definitions  based  on  missions  assigned  to 
special  operation  forces.  The  first  step  in  finding  potential  tools  was  to  examine 
the  existing  traditional  warfare  tool  suite.  Not  surprisingly,  artful  use  of 
traditional  M&S  tools  yields  some  very  limited  insights  into  irregular  warfare 
operations  that  incorporate  direct  action.  As  expected,  none  of  the  traditional 
tools  examines  all  aspects  of  the  irregular  warfare  mission  space.  Since  most 
irregular  warfare  missions  are  only  20%  military,  the  remaining  80%  of  the 
problem  —  political,  military,  economic,  sociological,  informational,  and 
infrastructure  (PMESII)  —  are  not  addressed  by  traditional  M&S  tools. 
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WG  1  -  Unconventional  Tools 
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Next  we  added  in  “unconventional  tools”  both  real  and  potential,  and  matched 
them  to  the  mission  areas.  Irregular  warfare  modeling  and  simulation  tools  can 
be  applied  in  kinetic  (Direct  Combat)  and  non-kinetic  (PMESII)  arenas  at  most 
levels  of  analysis.  Note  that  engineering,  system,  and  mission  level  models  are 
not  very  applicable  to  irregular  warfare  analysis.  In  general,  at  the  tactical  level 
of  analysis,  JCATS  is  a  world  class  model  for  examining  irregular  warfare  issues 
dealing  with  direct  combat.  To  achieve  several  of  the  mission  areas  stated  in  the 
WG  charter,  JCATS  may  have  to  be  implemented  in  unique,  non  traditional  ways 
or  be  augmented  with  other  models.  For  example,  riverine  operations  ... 
primarily  a  type  of  direct  operations  ...  may  be  analyzed  through  the  use  of 
GCAM  and  JCATS.  At  the  operational  level  of  analysis,  DIAMOND  US 
addresses  some  non-kinetic  issues  and  adequately  addresses  kinetic  issues. 
However,  the  model  was  designed  to  accommodate  a  medium  sized  problem  such 
as  Sierra  Leone  or  Liberia  and  is  incapable  of  handling  large  problems  due  to 
softw'are  limitations.  There  are  no  current  models  at  the  campaign  level  available 
to  analysts  that  adequately  examine  both  kinetic  and  non-kinetic  issues.  The 
commercial  wargame  derivative  could  be  used  to  assist  a  seminar  w'argame,  but 
is  unable  to  stand  on  its  own  due  to  aggregation  and  other  issues.  US  .Army 
CAA’s  ER1S  campaign  model,  when  it  is  complete,  may  provide  the  needed 
functionality  to  address  unconventional  warfare  and  counter  insurgency  issues. 
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One  promising  model  that  addresses  both  kinetic  and  non  kinetic  issues  is  PSOM. 
However,  it  is  also  used  to  inform  seminar  wargames  and  is  not  yet  ready  for 
field  use.  In  the  non-kinetic  arenas  of  Information  Warfare  and  Psychological 
Operations,  a  number  of  potential  tools  are  available.  A  number  of  agent  based 
models  could  be  used  to  examine  these  issues  but  suffers  from  formal 
accreditation  issues.  SOFBAT  has  a  high  potential  but  relies  upon  yet  to  be 
determined  information  operations  JMEM  like  information. 

There  are  several  caveats.  Other  than  JCATs,  almost  all  of  these  models  have 
never  been  used  in  a  formal  DoD  mandated  study  addressing  irregular  warfare 
issues.  As  a  result,  the  applicability  of  these  models  to  this  problem  area  may  be 
less  than  initially  perceived.  The  OOTW  tool  box  (comprised  of  Pythagoras, 
ISSM.  and  DIAMOND  US)  has  been  applied  to  J8  WAD’s  ESS  study  effort.  The 
remaining  tools  provide  potential  approaches  to  irregular  warfare  analysis.  Most 
PMESII  oriented  models  are  prototypes  (MIT  Systems  Dynamics  Model,  Agile, 
Commercial  Wargame  Derivative/IRIS-COINS)  or  have  been  applied  as 
COCOM  decision  making  aids  (e.g.  SEAS,  IBC).  Some  tools  are  dedicated  to 
GWOT  issues  e.g.  IGS/EBW.  SIAM  was  applied  to  an  exploratory  study  issue, 
but  has  not  been  used  for  the  formal  analysis  of  irregular  warfare  issues.  As 
mentioned,  SOFBAT  has  not  been  used  due  to  a  lack  of  data.  Only  DIAMOND 
has  been  used  to  size  UK  peace  support  operations.  Others  are  at  various  stages 
in  the  development  process  e.g.  PSOM  in  final  testing;  ERIS  in  design.  A  Force 
More  Powerful  has  just  been  released  for  sale. 
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WG  1 

-  What  Needs  to  be  Done”  — ■*— — 

* 

Frameworks/paradigms  for  Irregular  Warfare  Analysis 

• 

Irregular  Warfare  M&S  Evaluation  Criteria 

* 

Test  Cases 

* 

Irregular  Warfare  Scenarios 

# 

MOEs 

* 

Data 

* 

Irregular  Warfare  M&S  at  the  Campaign  Level 

* 

W&A 

* 

Analyst  (and  Consumer?)  training 

* 

Prototype  cross-cutting  studies 

• 

Best  of  breed  approach  to  model  improvement 

The  examination  of  the  wide  range  of  possible  models  produced  several  issues 
with  respect  to  how  the  models  could  be  used,  evaluated,  or  evolved.  However, 
the  problem  must  be  addressed  as  an  entire  system  of  analysts,  frameworks, 
methodologies,  test  cases,  scenarios,  evaluation  criteria,  data  and  MOEs.and  not 
just  one  of  modeling  and  simulation  tools. 

The  problem  in  analyzing  irregular  warfare  issues  are  usually  limited  by  the  lack 
of  an  analytical  framework  or  paradigm  by  which  to  analyze  complex 
phenomena.  The  lack  of  a  too]  or  several  tools  is  less  important  than 
understanding  the  phenomena  to  be  analyzed.  Direct  action  and  strategic 
reconnaissance  issues  are  essentially  traditional  combat  and  are  very  well 
understood.  As  mentioned,  insurgency  and  counter  insurgency  issues  are  only 
20%  military  and  80%  other  PMESII  variables.  Fortunately,  O’Neill,  SWORD, 
and  May  and  Neustadt  provide  frameworks  for  analyzing  these  problems. 
Distributed  operations,  shaping,  information  operations,  and  possibly  GWOT  are 
less  fortunate  in  this  respect. 

Traditional  criteria  for  evaluating  combat  simulations  deal  with  internal  validity, 
usability,  adaptability,  and  replication  of  results  (Robert  Axelrod  “Advancing  the 
Art  of  Simulation  in  Social  Sciences”  1997).  Although  these  criteria  also  apply 
to  irregular  warfare  models,  there  may  be  other  criteria  that  should  be  uniquely 
applied  to  this  class  of  models  to  evaluate  whether  one  model  is  better  than 
another. 
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The  curriculum  used  to  train  military  operations  research  analysts  may  have  to  be 
re-examined.  To  make  the  best  use  of  the  techniques  provided  by  other 
disciplines,  military  OR  analysts  may  have  to  receive  training  in  quantitative 
social  science  methods.  Perhaps  an  overview  of  historical  analysis 
methodologies  would  also  be  useful. 

This  examination  of  model  goodness  is  hampered  by  a  lack  of  scenarios  and 
studies  against  which  to  apply  these  tools.  As  mentioned  previously,  only  ESS 
has  examined  the  applicability  of  these  tools  against  a  specific  scenario. 
Additional  prototype,  cross-organizational  studies  should  be  undertaken  to 
evaluate  the  existing  list  of  tools  based  upon  approved  scenarios  provided  under 
the  analytical  agenda  process.  At  present,  there  are  few  analytical  agenda 
scenarios  that  cover  the  depth  and  breadth  of  irregular  warfare.  These  studies 
would  also  reinforce  a  best  of  breed  model  development  practice  to  achieve 
better  models  to  address  issues  in  the  irregular  warfare  arena. 

There  are  multiple  technologies  that  may  be  of  assistance  in  examining  irregular 
warfare  including,  but  not  limited  to  agent  based  models,  systems  dynamics,  and 
quantitative  computational  social  sciences.  Which  application  of  one  or  more  of 
these  techniques  to  irregular  warfare  problems  will  produce  the  best  answer  is  not 
clear.  Hence  a  best  of  breed  approach  is  needed  to  develop  better  models, 
especially  at  the  campaign  level  of  analysis.  The  DoD  should  fund  several 
different  approaches,  potentially  building  on  the  preliminary  DARPA  efforts. 
Various  tools  should  be  trialed  in  prototype  studies  and  the  results  should  be  fed 
back  into  the  tool  development  process. 
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WG  1  -  New  Stuff 


m©as 

MMttrY  DpelboM  Sortaty 


•  Games  as  tools 

♦  What  makes  a  game  helpful? 

♦  Scenario-based  planning,  seminar  wargames,  MMGGs? 

•  Learn  from  other  communities 

♦  Computational  social  science 

♦  Commercial 

**  Interface 

^  Usability,  flexibility,  agility 
Focus 

•  Data  mining  tools 

♦  Figure  out  what  they  Ye  good  for 


While  we  didn't  identify  any  “golden  BETs”  in  the  working  group,  several  concepts  did 
come  up. 

First,  while  games  have  been  used  by  the  military  for  many  years  to  consider  various 
issues,  we  thought  that  games  and,  broadly,  gaming  techniques,  might  be  better  able  to 
address  some  of  the  issues  that  come  up  m  dealing  with  populations  and  human  factors. 
By  “games"  we  mean  wargames,  to  include  scenario-based  planning,  seminar  wargames, 
conventional  wargames,  and  commercial  games.  We  feel  there  is  research  needed  to 
determine  what  the  applicability  is  of  various  gaming  techniques  to  the  problems  we  were 
considering. 

Second,  we  heard  a  lot  from  the  computational  science  community.  It  is  a  community 
that  has  wrestled  with  these  problems  for  many  years,  and  has  developed  theoretical  and 
practical  approaches  to  these  issues.  In  talking  to  members  of  this  community  during  the 
WG  session,  it  became  apparent  that  we  may  need  to  listen  more  than  we  talk,  in  order  to 
understand  their  practices  and  capabilities  from  their  perspective,  not  ours  as  operations 
researchers  or  physical  scientists. 

Another  community  identified  that  we  may  learn  something  from  is  private  industry. 
Companies  use  games,  models,  and  simulations  as  tools,  but  they  may  use  them  in 
different  ways.  The  interfaces  may  be  different,  the  way  they  are  procured,  built,  and 
evaluated  may  be  different,  and  the  focus  of  the  design  may  differ  from  DoD  products. 
Understanding  what  these  differences  are.  and  whether  they  could  be  incorporated  into 
DoD  practices  may  be  a  useful  area  of  enquiry. 

Finally,  we  heard  from  at  least  one  model  that  used  data  mining  as  a  way  to  develop  its 
input  database.  It  would  mine  the  internet  for  various  items,  then  accumulate  them  in  a 
database  to  be  used  in  a  simulation.  Data  mining  is  not  new,  and  there  are  many  data 
mining  systems  available.  It  may  be  worth  exploring  the  wide  range  of  ways  that  data 
mining  might  be  used  to  address  these  issues. 
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WG  1  -  A  Few  Suggestions 
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•  Change  in  the  consumption  of  the  analysis  may  be  the 
“killer  app” 

♦  Develop  ways  to  deal  with  uncertainty  in  the  inputs  and  results 

♦  Develop  ways  to  account  for  and  balance  risk  involved  in  different 
experimental  approaches  and  outcomes 

•  Consider  expanding  the  role  of  wargames  as  tools  that 
support  decision  making 

♦  But  they  have  to  be  done  right 

♦  And  you  have  to  balance  them  with  models 

•  Learn  to  use  range  of  outcomes  provided  by  multiple 
independent  analytical  approaches 


In  addition  to  recommendations,  and  new  ideas  for  tools,  we  tried  to  highlight  the  key 
points  and  issues  we  came  up  with.  There  were  three; 

1 .  Changing  the  way  analysis  is  consumed  may  be  the  most  important  and  cheapest  to 
implement  insight  we  can  provide  from  this  working  group.  It  may  also  be  the 
hardest  to  implement.  It  would  require  that  consumers  of  analysis  develop  ways  to 
deal  with  uncertainty  in  the  inputs  and  results.  It  would  also  require  that  the  analyses 
be  conducted  in  ways  that  balance  risk,  and  develop  multiple  possible  experimental 
approaches  and  outcomes.  This  is  something  that  may  be  worth  discussing  among 
analysts  and  analysis  consumers.  What  level  of  uncertainty  is  too  much  in  the 
results?  How  do  you  make  decisions  using  outcome  landscapes  instead  of  single 
instance  solutions? 

2.  One  topic  that  came  up  repeatedly  in  the  meeting  was  the  subject  of  expanding  the 
role  of  wargames  as  ways  to  support  decision  making.  Simulations,  particularly 
simulations  used  to  help  frame  and  focus  procurement  decisions,  play  both  an 
analytical  and  a  socialization  role  within  the  decision  making  process.  For  topics  that 
do  not  lend  themselves  to  simulation  or  modeling,  games  might  play  a  more 
substantial  role  in  that  decision  making  socialization  process. 

3.  Another  recurring  topic  was  the  need  to  have  a  range  of  theories,  models, 
methodologies,  and  outcomes  available  when  assessing  a  particular  problem  related  to 
human  factors.  We  were  briefed  on  a  recent  DARPA  effort  that  involved  multiple 
competing  models  that  examined  government  stability.  The  results  of  examining  the 
landscape  of  model  solutions  provided  greater  insight  then  that  of  an  individual 
model.  This  concept,  of  using  multiple,  different  approaches  to  address  the  same 
problem  was  felt  to  be  more  appropriate  for  these  problems  than  the  use  of  one, 
individual  model  or  approach. 
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WG  1  -  Recommendations 
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•  Getting  the  question  and  the  framework  right  is 
important  ■  and  relatively  inexpensive 

♦  But  it  can  take  time,  and  iteration 

•  Examine  the  possibility  of  collecting  reai-world  data  to 
support  decisions 

•  Examine  groups  of  models 

•  Research  the  use  of  games/scenario  planning/ 
commercial  techniques  to  support  decision  making 


The  WG  recommendations  are  contained  on  this  slide  and  the  next.  They 
summarize  what  has  come  before  and  expand  on  these  concepts. 

While  getting  the  question  right  is  important,  it  is  also  important  to  acknowledge 
that  it  can  take  time  and  involve  the  sponsors  in  an  iterative  process. 

There  is  a  need  for  real-world  data  to  support  analysis  and  modeling.  It  may  be 
useful  to  examine  the  various  impediments  to  collecting  this  data  including  legal, 
programmatic,  and  other  issues. 

As  we  mentioned,  examining  an  ensemble  of  models  may  be  important  to  give 
decision  makers  a  landscape  of  solutions. 

Finally,  it  maybe  fruitful  to  examine  various  wargaming  and  commercial 
techniques  as  they  apply  to  addressing  these  problems. 

While  we  did  not  find  answers  to  the  problems  we  were  presented  with,  we 
believe  we  have  identified  some  specific  actions  that  could  be  taken. 
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WG  1  -  Recommendations 
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•  Examine  new  ways  to  validate  models,  and  find  ways  to 
deal  with  what  traditionally  cannot  be  validated 

♦  Any  model  that  deals  with  human  decision  making 

♦  While  they  ca  not  be  validated  they  can  be  used  to  inform  and 
support  decision  making 

♦  Traditional  assumptions  of  reproducibility  and  traceability  may  need 
to  be  replaced  with  transparency  and  acceptance  of  judgment 

•  Examine  thinking  about  the  role  of  modeling,  and  the 
need  for  scientific  answers  for  questions  that  may  have 
no  basis  in  physical  science 

♦  May  need  to  accept  more  risk  in  this  area 


One  topic  that  was  discussed  throughout  the  working  group  was  the  question  of 
how  models  and  other  techniques  should  be  validated.  It  may  be  that  traditional 
methods  used  for  validation  may  not  apply  to  the  issues  discussed  in  this  working 
group.  .We  need  to  work  to  understand  what  can  and  cannot  be  .validated  and 
what  can  be  used  even  though  it  cannot  be  validated  through  the  traditional 
mechanisms.  Finally,  it  may  be  important  to  examine  traditional  assumptions 
about  reproducibility  and  traceability  that  are  required  for  valid  operational 
analysis  when  considering  subjects  like  human  behavior,  population  dynamics, 
and  other  soft  questions. 

Ultimately  the  question  of  validation  comes  down  to  the  role  of  the  decision 
maker  and  the  degree  of  risk  they  are  willing  to  assume  in  making  a  decision 
based  on  information  that  is  not  reproducible  or  scientific  in  the  physical  sciences 
sense.  In  the  discussions  that  went  on  in  the  working  group,  it  became  apparent 
that  many  of  the  questions  that  decision  makers  may  ask  may  not  be  answerable 
through  traditional  operations  research  techniques.  The  challenge  may  be  to 
develop  new  ways  of  dealing  with  that  reality  —  ways  that  still  provide  a  robust, 
rigorous,  and  credible  ability  to  answer  the  questions. 


27 


SWG  1-1  —  Worlds  Collide 
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♦ 

♦ 


Liberal  arts 


Interagency 

[  Analyst  ) 

OR/Physica!  Science 


What  are  the  right  ingredients  to  turn  the  social  science 
into  a  campaign  level  analysis? 

Each  is  uncomfortable  in  the  others  world 
Need  to  begin  to  merge  disciplines 


Sub-Working  Group  One  (SWG  1-1)  was  asked  to  examine  the  ability  of  analysis 
and  models  to  guide  an  influence  campaign  designed  to  discourage  popular 
support  for  an  irregular  force,  and  deter  the  force  itself. 

One  realization  that  came  out  of  that  group  was  that  there  are  different  worlds 
that  deal  with  different  aspects  of  the  military  planner’s  problem.  The  analyst 
exists  in  the  operations  research/physical  science  world.  With  that  comes  a 
world-view  and  concept  of  analysis  that  can  sometimes  be  at  odds  with  the  view 
of  the  social  scientist.  Similarly,  the  interagency  is  often  composed  of 
individuals  who  comes  from  a  different  background,  in  political  science,  for 
example.  The  military  planner  and  decision  makers  also  come  from  an 
operations  and  often  an  engineering  background.  In  dealing  with  irregular 
warfare  issues,  however,  it  will  be  important  for  all  of  these  different  worlds  to 
work  together  toward  a  common  understanding  of  how  to  approach  and  analyze 
the  problem.  The  feeling  in  this  sub- working  group  is  that  as  all  of  these 
different  entities  begin  to  work  together  toward  a  common  understanding  of  the 
problem,  analysis,  and  tools  —  the  fields  will  also  need  to  begin  to  work  together 
collaborative]  y. 
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I  CAN’T  STAND  THIS 
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Scenario:  Kazakhstan,  radical  Islamic  insurgents,  Russians 
on  the  ground  with  a  heavy  hand  on  the  local  population 

Objective:  Work  with  Russian  counterparts  to  develop 
coordinated  elements  of  a  public  diplomacy,  information 
operations  and  TSC  campaign  to  thwart  burgeoning  spread 
of  radical  Islam  to  the  Kazakhstans 


The  first  sub-working  group  to  brief  out  was  asked  to  examine  a  complex 
scenario  involving  Kazakhstan  and  a  radical  Islamic  insurgency,  The  key 
objective  for  the  group  was  to  identify  how  analysis  could  support  the  COCOM 
in  developing  a  plan  for  public  diplomacy,  information  operations,  and  theater 
security  cooperation. 
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Customer:  COCOM,  with  potential  diplomatic  overlap 

Time  Horizon:  Undefined,  but  not  immediate 

Problem  definition:  Provide  analytic  support  to  COCOM 
staff  to  help  plan  a  variety  of  flexible  deterrent  options 
(FDOS)  covering  the  growing  radicalization  of  the 
Kazakhstans 

CPB:  Network  of  relationships,  model  -  not  simulation, 
political  map,  cultural  map 


This  was  the  way  the  working  group  parsed  the  problem  and  the  timeline. 

Analysis  was  seen  to  be  able  to  help  with  an  understanding  of  the  battlefield. 
This  was  termed  Cultural  Preparation  of  the  Battlefield  (CPB). 

First  the  group  wanted  to  identify  the  relationships  between  various  individuals 
involved  in  the  insurgency. 

Then  they  wanted  to  develop  a  model,  of  the  insurgency.  This  was  not  a  model 
in  the  sense  of  a  simulation,  but  rather  a  way  to  think  about  the  relationships  and 
cultural  factors  that  went  into  the  insurgency. 

Finally  they  wanted  to  develop  two  maps  that  would  describe  the  insurgency,  a 
political  map  and  a  cultural  map. 
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Wargame:  Transparency,  communication,  analytical 
challenge  and  alternative  theories 

Assessment:  Polling,  selected  interviews 

Contingency:  Traditional  models,  CPB  and  wargame 
based  assumptions 


The  sub-group  felt  that  the  best  way  to  identify  many  of  the  issues  involved  in 
the  scenario  was  through  conventional  wargaming.  It  would  provide  alternative 
theories  of  what  courses  of  action  to  take,  and  be  relatively  transparent  to  those 
involved. 

« 

Assessing  the  courses  of  action  would  require  polling  and  interviews  of  the 
population  in  order  to  establish  ground  truth  data. 

This  effort  could  feed  into  planning  for  a  military  contingency  operation,  which, 
if  needed,  would  benefit  from  the  cultural  preparation  of  the  battlefield. 


31 


C’EST  LA  VIE!  -  Hard  Issues 


m©H5 

MUnzry  OpnDoM  ftft  SoOtty 


•  Will  the  violence  continue  to  escalate? 

♦  Trend  analysis  of  web  text  searches  such  as  Biackiight;  Public  poll 
analysis  could  also  be  useful 

•  Can  we  predict  the  population  dynamics  prior  to  mass  insurgency 

♦  Could  gain  insights  into  dense  recruitment  pools/areas  by  combining 
multiple  data  sets  (age  demographics,  unemployment  levels,  crime 
rates,  perceived  inequalities,  income  levels,  education  levels, 
election  results,  culture) 

■» Combine  these  data  sets  using  map  overlays  or  GIS  tools  to 
create  visual  recruitment  landscape 

Landscape  could  prove  useful  for  Operation  planning  (Focus  10 
on  dense  populations  of  potential  recruits) 

•  Is  it  possible  to  model  COIN  in  a  Western  nation? 

♦  PSOM  with  changes  (Increase  resolution  in  urban  areas) 


This  sub-working  group  was  tasked  with  analyzing  a  Canadian  scenario  with  an 
emergent  insurgency  in  the  province  of  Quebec. 

They  started  off  with  what  they  thought  were  the  hard  issues,  ones  that  did  not 
•  have  a  clear  analytical  solutions  path. 

One  thing  they  realized  was  that  this  scenario  highlighted  the  difference  in 
available  data  between  an  open,  post-industrial  nation  such  as  Canada  and  less 
developed  nations  or  ones  that  do  not  have  as  free  expression  as  citizens  of  libera! 
democracies.  In  this  scenario  the  group  felt  that  data  mining  tools  would  be 
important,  including  the  Biackiight  tool  the  group  had  been  briefed  on  during  the 
plenary  session.  Likewise  polling  and  other  data  would  be  both  available  and 
more  reliable. 

When  combined  with  demographic  data,  the  group  felt  that  a  data  landscape 
could  be  created  using  GIS  or  other  tools  that  would  provide  operational  planning 
capabilities,  whether  for  10  or  to  identify  areas  of  potential  unrest. 

One  tool  that  was  briefed  to  the  plenary  session  that  this  group  felt  would  be 
useful  was  the  PSOM  tool.  With  changes  it  could  be  used  to  possibly  model 
COIN  operations.  The  changes  would  involve  decreasing  the  size  of  the 
calculated  grid  squares  so  that  the  higher  population  density  of  an  urban  area 
could  be  assessed. 
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•  Are  there  models  of  border  flows 

♦  Traffic,  transportation  and  immigration  authorities  could  potentially 
provide  insights/models 

♦  Terrain  analysis  could  also  be  very  useful 

•  What  aid  could  the  US  provide  if  asked? 

♦  Waterway  and  critical  infrastructure  protection 

■*  Modeled  using  conventional  kinetic  models  and  network  models 
of  the  power  grid 

♦  Border  protection  and  arms  interdiction 

•* ISR  Models,  Kinetic  models 

•  Historical  models  of  Ireland,  Kosovo,  and  1980  Quebec 
insurgencies  could  prove  useful 


These  are  the  group  of  issues  that  the  sub  working  group  decided  could  be 
handled  by  existing  models  or  simulations.  The  issues  included  modeling  or 
accounting  for  possible  cross  border  flows  of  people  or  weapons.  Here  existing 
models  from  other  agencies  might  be  pulled  into  the  analysis.  US  aid  to  Canada 
could  be  assessed  by  using  straightforward  military  models  that  draw  on 
conventional  kinetic  or  ISR  models. 

Finally,  the  group  felt  that  historical  models  of  past  insurgencies  might  prove 
useful  in  guiding  the  development  of  courses  of  action.  The  use  of  historical  data 
to  assist  in  model  validation,  or  in  providing  insights  into  insurgencies,  was  a 
theme  that  recurred  throughout  the  working  group. 
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C’EST  LA  VIE!  -  Insights 
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•  Modeling  COIN  in  Western  nations  is  significantly  different  than 
doing  so  in  other  areas 

♦  Availability  and  credibility  of  data  allows  for  better  analysis  of  current 
news  sources 

•  True  simulations  seem  to  fall  short,  but  there  exists  value  in 
methodologies  that  combine  relatively  simple  algorithms  with 
strong  data  resultant  of  free  speech 

•  Multiple  tools  that  provide  varying  insights  are  more  valuable  than 
the  search  for  one  tool  that  claims  to  determine  an  optimal 
solution 

•  US  action  could  potentially  free  Canadian  forces  for  use  on 
operations  where  they  possess  a  competitive  advantage  (OPS 
requiring  local  knowledge;  Troops  in  the  streets) 

♦  This  could  allow  the  US  to  maintain  a  low  profile  and  prevent 
potential  Canadian  civilian  backlash 


This  group  identified  two  key  themes  that  relate  to  our  findings  from  the  plenary 
sessions: 

1.  Data  availability  can  provide  additional  ability  to  model,  and  do  analysis, 
when  its  available.  Because  a  larger  set  of  demographic,  cultural,  and 
informational  data  was  available  in  this  scenario,  a  wider  range  of  analytical 
tools  and  techniques  could  be  tried.  And  those  techniques  would  also  have 
a  better  chance  of  accurately  reflecting  reality. 

2.  The  use  of  multiple  tools  would  provide  the  commander  the  ability  to 
evaluate  a  wide  range  of  theories  and  principles  in  making  decisions, 
instead  of  trying  to  identify  one,  optimal  solution,  For  insurgencies,  such  a 
solution  might  not  be  capable  of  being  identified  through  traditional 
operations  research  analysis. 
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Other  Insights  and  Random  Thoughts 

■  D  ARP  A  P  CAS  prog  ra  m  seems  p  romi  sing 

♦  There  is  a  concern  that  the  program  is  terminating  or  receiving  a  low  level  of  funding 

•  Use  of  low-complexity  vs,  high -complexity  modeling  In  DoD 

♦  Are  we  using  high-definition  modeling  because  we  have  them  and  know  how  lo  run  them, 
instead  of  directing  analyses  reward  the  tough  issues  DoD  decision  makers  face 

♦  Cease  and  desist!  Set  our  analytical  talent  on  addressing  the  key  issues  and  tel  them  figure 
out  how  to  do  it 

•  Are  we  tapping  the  vast  world  of  analysis  of  social  science  phenomena  In  academia  and 
business?  When  we  do,  can  the  techniques  that  we  find  create  useable  tools  for  our 
decision  makers? 

•  Should  the  analytic  community  shift  some  of  its  resources  toward  problem  definition? 

♦  Many  questions  ma  y  be  amena  ble  to  qua!  itative  ana  lysis  a  nd/or  s  im  pie  a  na  lytic  techn  iques 
vice  complex  modeling 

•  Is  MORS  the  problem?  Is  our  discipline  built  too  much  around  classical  OR  (the  WW-lt 
legacy)?  Need  more  analysts  to  be  trained  In  Social  Science  modeling.  Find  ways  to  bring 
the  work  that  is  being  done  Into  the  decision-  and  policy-making  levels  Tap  into  fields  like 
anthropology  that  are  not  well -represen ted  in  our  analytical  community, 

•  We  ought  to  be  doing  OA  studies  at  the  inter-agency  level.  Encourage  MORS  participation 
by  oon-DoD  agencies 
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The  last  working  group  featured  in  the  briefing  was  asked  to  examine  military 
operations  in  urban  terrain  (MOUT).  We  included  their  last  slide  as  the  last  slide 
in  the  briefing  because  it  emphasizes  some  points  that  have  already  been  made: 

•  Using  multiple  models  as  was  done  in  the  DARPA  PCAS  project  to 
develop  solution  landscapes,  One  way  to  do  that  is  to  develop  many  low- 
definition  models  instead  of  a  small  number  of  high-definition  models 

•  Involving  the  social  sciences  in  helping  develop  our  analytical  tools. 

•  Examine  the  problem  of  defining  what  the  problem  is.  Perhaps  additional 
work  is  needed  in  understanding  exactly  what  is  being  asked  of  the  OR 
community. 

This  group  also,  however,  identified  several  other,  important,  insights  into  the 
problem  of  analyzing  irregular  warfare: 

•  Throughout  the  conference  the  role  of  interagency  participation  was 
discussed,  but  never  examined  in  detail.  Analysis  of  interagency-related 
issues  and  problems  could  be  a  major  conference  by  itself. 

•  Is  MORS  the  problem?  Should  we  open  the  field  of  operations  research 
to  a  broader  and  more  diverse  group  of  individuals  who  can  bring 
techniques  and  talents  that  we  currently  do  not  have  available? 
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WG  1  -  Charter  and  Approach 


me*5 

HWtMV  OpidDgni  tawnh 


•  GWOT  and  Irregular  Warfare 

♦  A  large  set  of  topics 

♦  The  largest  session  {80  people) 

•  Approach 

♦  Baseline  with  stakeholders 

♦  Critically  examine  existing  models 

♦  Search  for  new  ideas  in  the  best  places 

♦  Academia,  FFRDC's,  gamers 

♦  Involve  everyone  in  small  groups 

•♦Using  a  scenario-based  approach 

♦  We  have  interesting  speakers,  plan  to  introduce  stimulating 
ideas 


t 


37 


Details 
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•  Baseline  participants 

♦  Session  overview,  common  set  of  questions 

♦  How  do  we  account  for  what  we  currently  don't  account  for? 

•  Scenarios 

♦  Designed  to  stimulate  discussion,  provide  interest 

♦  Will  be  facilitated  discussion 

♦  10  Scenarios  -  will  modify  according  to  number  of 
participants  to  keep  7-10  people/group 

♦  Scenarios  range  from  planning  and  resourcing  to  execution 

♦  All  are  focused  on  addressing  the  hard  problems 
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«  Day  1:  Stakeholder  Briefings  and  Round  Table 

*  OSD  PAS.E 

Mr.  Preston  Duniap 

*  US  Air  Force  No  Representative 

*  US  Army 

No  Representative 

*  SOCOM  Mr.  Ed  Cardenas 

*  US  Navy 

Captain  Andy  King 

*  EUCQM  No  Representative 

•  US  Marine  Corps 

Mr.  Steve  Stephens 

*  CENT  COM  No  Representative 

•  Day  2: 

*  DoD  La rge-Scale  Mod e  1  Brief! ng s  a nd  Roundta ble 

«*  Larry  Budge 

A  Won 

SEAS 

+  Ed  Cardonas 

NG/SOCOM 

sofrat 

■*  Tom  Couture 

Group  W 

MOOTW  FAST  Toolbox  ( ISSM t  JCATS t  Diamond  US) 

■+  Nicola  Stewart 

UK  MOD 

PSQM  Peace  Support  Operations  Model 

<*  John  Shepherd 

US  Army  CAA 

E/US  (JOES  Cf) 

*  Larry  Budge 

Alton 

The  Sentient  World 

*  Novel  Approaches  Briefings  and  Roundtable 

■+  Mr.  Martin  Udy 

IDA 

Interdepartmental  Analytical  Framework 

+  Dr.  Robert  Popp 

darpa 

Quantitative  Computational  Social  Sciences 

■+  Douglas  Whatley 

Breakaway 

A  Force  More  Powerful 

4  Mr  Aaron  Frank 

BAE 

Quantitative  Computational  Social  Sciences 

■*  Ken  Comer 

OSDPA&E 

W&A:  Agent  Based  Models 

■+  Dr.  Alok  Chataverdt 

SIMULEX 

W&A:  Agent  Based  Model's 
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WG  1  -  Draft  Schedule  (Concluded)  Ii1©j)5 

Hill t*t- y  Operatic**  I Mhiy 

*  Day  2: 

*  Sub  Working  Group  Scenario  Discussions  during  Lunch 


Or,  Ed  McGrady 

CNA 

1  Economics  101 

Mr.  Ed  Cardenas 

SOCOM 

2.  Up  the  River 

Dr.  Cy  Stanit 

NG 

3.  You  Musi  Be  On  Drugs 

Mr.  Sieve  Stephens 

USMC 

4.  /  Can't  Stand'  This 

LTC  Sieve  Krrighf 

JB  WAD 

5.  /  Wanna  go  back  ro  the  islands 

CMDR  Tom  McNemey 

USN 

6.  Pirates  are  Cool 

Mr  John  Cottier 

OSD  PA&E 

7.  C  ast  La  Vie! 

Mr  Mike  Williams 

OSD  PA&E 

Mike  Ortenbeng 

OSD  PA&E 

3 ,  A  Dispute  of  Long  Standing 

Mr  John  Snevely 

OSDPA&E 

9  A  Place  in  the  Sue 

Mr.  Mike  FltiSimmons 

IDA 

10.  MOUT  +  Terrorists  *? 

Dr  Royce  Knee ce 

IDA 

■  Day  3 

*  Core  focus  out-brief  prepared 
+  Working  groups  prepare  out -briefs  on  scenario's 
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Analysis  for  Non-Traditional 
Security  Challenges: 
Methods  &  Tools 

Military  Operations  Research  Society 
(MORS)  Workshop 

WG  3  -  Catastrophic 
Outbrief 
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WG  3:  Overview  mens 


HLD;  State  of  data, 
methods,  tools 

♦  Accelerate  existing  programs  focused  on  engineering  tactical  model 
integration  Wot  ready  to  build  new  operational  or  strategic  models: 
focus  on  data  collection  from  exercises,  operations  and  war  games 

♦  Conduct  analytically  supported  war  games  to  clarify  roles,  concepts, 
COWOPS,  and  metrics  to  better  refine  needs 

♦  Facilitate  or  inter-agency  analytical  Community  of  Interest  (COI) 

♦  HLD  imperative:  Born  joint-* *  Born  interagency/international 


Irregular 


Catastrophic 


Traditional 


Disruptive 


Workgroup  3  Coverage 


WG  3  (Catastrophic  )  did  well  assessing  the  status  of  analysis  and  M&S  for  homeland  defense 
(particularly  consequence  management),  but  less  so  for  counter  WMD, 

Per  the  Model  Hierarchy,  the  community  appears  to  have  a  large  collection  of  data,  methods,  and  tools 
in  use  for  engineering  and  systems  levels  and  a  few  at  the  mission  and  tactical  levels,  but  there  are 
almost  no  applications  at  the  operational  or  strategic  levels.  In  other  words,  there  is  no  single 
application  that  can  be  used  to  model  the  homeland  defense  or  counter  WMD  campaign. 

Current  tool  and  method  development  is  focusing  on  integrating  the  existing  tools  to  better  support 
mission  and  tactical  level  analytical  questions,  especially  to  support  quick-tum  response  for  COCOMs 
and  future  first -responders. 

For  analysis  at  the  operational  and  strategic  level,  the  community  is  not  yet  ready  to  define  the  new 
methods  or  tools  that  are  needed;  instead,  the  focus  is  on  data  collection  from  exercises,  operations  and 
war  games  to  better  understand  the  models  that  are  needed.  Data  may  be  collected  through  data  mining 
or  participation  in  wargames. 

Focal  activity  will  be  analytical  support  of  wargames  so  that: 

*  A  complex  set  of  interagency  roles  and  capabilities  can  be  identified  and  clarified 

*  A  best  practice  CONORS  of  all  role  players  can  be  identified  (first  responders,  local  governments, 
US  states/National  Guard,  DHS/FEMA,  EPA,  CDC/NIH,  DoD) 

*  A  Metrics  of  success  can  be  developed  (e,g,,  time  to  respond,  lives  saved,  etc,) 

A  key  way  ahead  for  the  WG  is  to  develop  a  robust  CGI  to  keep  DoD  participants  aware  of  all  DoD 
activities  but  also  as  a  way  to  collaborate  and  coordinate  with  other  agencies  on  a  routine,  action  level 
basis;  COIs  might  use  a  website/blog  forum  to  facilitate  the  exchange  of  ideas  or  documents 

Homeland  Defense  analysis  can  not  be  done  well  without  extensive  non-DoD  participation,  therefore  all 
HLD  activities  must  be  bom  interagency/international. 
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WG  3:  Strategic  Challenges  [?1SR5 

HtlUvr  Oimtforv  huHi  W*oty 

•  Catastrophic  homeland  events:  Man-made  or  naturally  occurring  events 
involving  significant  loss  of  life  and  significant  sodal/poiitical/ 
economic/psychological  effects:  e.g.  large  National  Planning  Scenarios 
(1  OKI  nuke,  chlorine  tank,  plague,  anthrax,  large  natural  disasters) 

♦  COI:  NORTHCOM,  PACOM,  Services,  DTRA,  National  Labs,  DBS  (includes  Coast 
Guard,  PEMA),  HSC,  EPA,  DOE 

•  Counter-WMD:  Events  involving  WMD  interdiction,  counter-proliferation, 
deter/defeat  the  use  of  WMD  by  state  and  non-state  actors 

♦  COI:  STRATCOM,  Services,  Regional  COCOMS,  DTRA,  National  Labs,  DOE 

•  Significance  = /'(Physical,  economic,  social,  political  and  psychological 
effects,  level  of  effort,  duration) 

•  Some  key  unresolved  issues 

♦  Rotes  and  relationships  (Lead  Federal  Agency,  Title  10/32,  authority) 

♦  Lack  of  coordination,  integration,  inter-operability 

♦  Immature  data,  methods,  tools  in  some  areas 


The  first  issue  we  had  to  decide  upon  was  what  constitutes  a  catastrophic  event. 
Our  first  cut  at  this  was  to  examine  the  15  National  Planning  Scenarios  developed 
by  the  Homeland  Security  Council  and  DHS.  From  these  15  scenarios,  there  are 
several  that  involve  significant  loss  of  life  and/or  significant  social,  political, 
economic,  or  psychological  effects.  These  scenarios  are  listed  and  would 
probably  necessitate  a  DoD  response. 

The  second  catastrophic  area  is  counter  WMD  and  our  definition  of  this  area  is 
shown  above.  We  feel  that  the  COI  for  this  area  is  a  bit  more  restricted  than  the 
homeland  defense  area. 

Regardless  of  which  of  the  two  major  areas  you  are  examining,  determining  the 
level  of  significance  of  the  event  can  be  defined  as  a  function  of  the  factors 
shown.  In  our  discussions,  we  recognized  that  the  political  and  psychological 
factors  may  be  dependent  on  current  events.  In  other  words,  events  that  occurred 
before  9-1 1  that  were  not  considered  catastrophic  at  that  time  may  now  be 
considered  catastrophic  when  viewed  through  the  lens  of  9-1 1. 

In  our  view,  there  are  still  some  thorny  unresolved  issues.  These  issues  generally 
fall  into  the  three  broad  categories:  1 )  roles  and  relationships;  2)  coordination, 
integration  and  interoperations;  and,  3)  immature  data,  methods  and  tools  in  some 
areas. 
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M&S  Capabilities  - - 

•  Current  capabilities 

♦  Engineering  level:  Robust  individual  models:  DTRA  Suite,  CAPS 

♦  Tactical/operational  level:  EPICS,  JCATS,  IWMDT 

♦  Campaign  level:  Legacy  models  (ITEM),  but  these  are  not  tailored  for  domestic 
catastrophic  events 

♦  Training  models:  Significant  lack  of  simulators,  stimulators,  and  facilities 

•  Applications/exampies  -  many  stand  alone  tools.  Multiple  tools  are 
often  required  to  assess  a  scenario 

♦  Defense  Collaboration  Tool  Suite/I  nfoWorkSpace  (DCTS/IWS) 

♦  Hazard  Prediction  Assessment  Capability  (HPAC} 

♦  Consequence  Assessment  Toolset  (CATS) 

♦  Integrated  Munitions  Effects  Assessment  (IMEA) 

♦  Vulnerability  Assessment  and  Protection  Option  (VAPO) 

♦  Traditional  campaign  analysis/mission  level  tools  re-purposed  to  assess 
catastrophic  scenarios 


The  majority  of  the  current  M&S  tools  and  capabilities  generally  fall  into  the 
engineering  and  mission  areas  of  the  M&S  pyramid  although  there  are  a  few 
tools  available  at  the  tactical  level.  There  is  some  effort  being  applied  to  use 
some  of  our  legacy  campaign  models  (such  as  ITEM)  in  the  HLS/HLD  arena,  but 
these  tools  are  generally  not  tailored  for  analysis  of  domestic  catastrophic  events. 

The  example  applications  shown  above  are  just  the  tip  of  the  iceberg  with  regard 
to  our  ability  to  model  WMD  effects.  These  tools  are  generally  very  good  at 
modeling  first  order  effects  and  the  spread  and/or  transmission  of  those  effects 
from  the  source  through  the  target  area.  Generally,  these  tools  are  also  stand¬ 
alone  products,  and  therefore  the  results  of  one  model/tool  cannot  directly  feed 
the  inputs  of  another  model/tool. 
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WG  3:  On-going  Enhancements  to 
Analytic  and  M&S  Capabilities 

•  Selected  engineering  and  system  level  tools 

♦  Joint  Effects  Model  (JEM):  Predict  and  track  NBC  and  Toxic 
Industrial  Chemical/Material  (TIC/TIM)  events  and  effects 

♦  Joint  Operational  Effects  Federation  (JOEF):  M&S  for  warfighters 
and  planners  to  accurately  predict  CBRNE  effects 

♦  The  Joint  Warning  and  Reporting  Network  (JWARN): 
Comprehensive  analysis  and  response  capability  to  minimize  the 
effects  of  NBC  attacks,  accidents  and  incidents 


DTRA  will  begin  to  spiral  Integrated  WMD  Toolset  (IWMDT)  FY06 


DTRA  Collaboration  Center  (DCC)  -  common  architecture  that 
supports  CBRNE  data  fusion  from  disparate  sources  supporting 
Situational  Awareness,  Systems  Engineering  and  capability 
development:  initial  operational  capability  FY06 


DTRA  partnering  with  Services,  COCOMS,  Allies,  Labs 


DTRA  is  currently  involved  in  enhancing  several  of  the  M&S  tools  to  better 
support  their  warfighter  customer.  These  tools  will  become  part  of  the  IWMDT 
and  will  be  available  through  a  reachback  function  centered  on  the  DTRA 
Collaboration  Center. 
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WG  3:  Analysis  Gaps  and  Deficiencies  m©H5 
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•  Key  stakeholders  and  decision  makers 

♦  USG  (Inter-agency)  ♦  DoDi'Servioes/COCOMS  ♦  Local  Governments 

♦  Allies  *  Coalitions  ♦Industry 

♦  State/TemtoriaJ  Govemments/NG  +  National/private  labs  *  Rrsl  responders 

•  Key  analysis  gaps  and  deficiencies 

♦  Perceived  Lack  of  objective  goals,  defined  responsibilities,  actions/resources  reflecting  stated 
priorities 

♦  Need  for  improved  interagency  cooperation,  collaboration,  and  data  sharing 

♦  Insufficient  understanding  of  complex,  primary  effects  and  secondary  effects  (political,  economic, 
social,  psychological) 

♦  Little  quantification  of  local,  state,  federal,  DoD,  and  International  response  capabilities 

♦  Immature  and/or  nonexistent  operating  procedures,  data,  toots  for  evolving  mission  and 
requirements 

♦  flamers  to  sharing  data,  plans  (classification  issues,  terminology,  ontology) 

•  Commonalities  between  stakeholders 

♦  Desine  for  comprehensive,  systematic,  interagency  analysis 

♦  Desire  for  integrated,  tactical/operational  tools 

♦  Need  to  plan  for  response  and  future  investments 

♦  Mission  focus 


To  address  the  list  of  gaps  shown,  the  analysis  work  needs  to  be  coordinated  among  or  dra  w  data  from  all 
stakeholders,  because  the  plans/CONOPS  for  each  stakeholder  will  be  influenced  or  dependent  on  that  of  the 
other  stakeholders.  Any  analysis  to  attempt  to  optimize  planning,  investments,  or  support  tools  for  consequence 
management  will  be  inaccurate  unless  these  stakeholder  interdependencies  are  captured  in  the 
mode  ling/ana  ly  si  s , 

The  perceived  lack  of  priority  listed  as  a  gap  is  not  an  indication  that  anyone  has  said  that  HLD  is  not  important 
but  that  other  priorities  have  let  roadblocks  to  effective  HLD  analysis  {e.g,t  natural  tendency  tow-ard  stove-piped 
activity)  remain  unresolved. 

The  lack  of  clear  goals  is  due  to:  1 )  a  lack  of  understanding  about  what  each  stakeholder  can  reasonably  hope  to 
achieve;  and,  2)  unfamiliarity  with  the  comprehensive  and  complex  effects  of  a  catastrophic  event,  e.g.}  don't 
understand  all  the  effects  that  have  to  be  managed  (economic,  political  etc.). 

Analytical  support  of  and  participation  in  exercises  and  wargames  appears  to  be  the  best  starting  point  for 
eliciting  the  data  that  will  clarify  stakeholder  roles  and  capabilities  for  HLD. 

Analytical  exercises/wargames  should  also  help  stakeholders  improve  on  the  immature  requirements  and  plans 
for  the  HLD  vice  the  military  theater  mission.  The  key  issue  is  that  the  DoD  has  typically  had  a  passive  role  in 
HLD /consequence  management,  i,e.,  provide  what  is  available  when  calls  for  help  come  in.  However,  based  on 
the  DoDs  expertise  in  systematic  planning  for  rapid  response  and  use  of  iterative  exercises,  wargames, 
modeling,  and  simulation  to  develop  and  refme  response  plans,  capabilities,  and  program  investments,  the  DoD 
needs  to  step  up  to  support  the  larger  interagency  community  in  developing  the  HLD  response  plans,  even 
though  the  DoD  will  likely  always  start  in  a  supporting  role  in  any  catastrophic  scenario  and  will  not  be  able  to 
dictate  a  complete  response  plan. 

Breaking  down  barriers  between  analysis  communities  within  the  DoD  and  between  agencies  will  be  facilitated 
by  a  standard  data  format  and  by  methods  for  navigating  around  classified  data. 

The  list  of  commonalities  among  the  entire  set  of  stakeholders  —  across  all  agencies  —  reflects  universal 
concern  about  catastrophic  scenarios.  There  also  is  strong  consensus  that  no  stakeholder  can  go  it  alone; 
Hurricane  Katrina  re-emphasized  that  we  are  not  ready  to  address  no- warning  scenarios  and  that  a  way  around 
and  through  traditional  stovepipes  must  be  found  and  cultivated  as  soon  as  possible. 
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•  Data 

♦  Lack  of  data  on  local  state,  other  federal  NGO,  int'l  agency  capability 

♦  Biological  threats,  retransmission,  mortality,  etc. 

♦  CORN  effects  (lack  of  cument/empirical  tesi  data) 

•  CONORS 

♦  Desired  response  over  time 

♦  Metrics 

♦  Translation  of  DoO  capabilities  to  HLD  employment 

*  Pre-event  (detect,  deter,  prevent  capabllilies) 

Post-event  (consequence  management) 

♦  Apportionment  of  tasks  across  services 

•  Methods/Tools 

♦  Model  integral  ion/synergy  (output  of  one  modal  not  always  useable  by  others  to  provide 
useful/ meaningful  output) 

♦  Catastrophic  models  (e  g  Biological) 

♦  Empirical  based  models 

♦  Socio-economical  psychological  and  political  aspects 

♦  integrated  M&S  “pyramid,"  particularly  at  higher  levels 


The  M&S  shortfalls  and  gaps  we  have  identified  can  be  summarized  in  three  areas  —  data,  CONOPS. 
and  methods/tools.  With  regard  to  the  gaps  in  the  data  area,  we  see  three  major  deficiencies: 

1 .  The  lack  of  data  concerning  the  capabilities  of  the  local,  state,  other  federal,  NGOs,  and 
international  agencies.  It  is  extremely  difficult  to  predict  what  capabilities  will  be  requested 
from  DoD  in  response  to  a  catastrophic  event  when  we  have  no  idea  of  what  portions  of  the 
response  activities  can  be  handled  before  DoD  is  requested  to  enter  the  fight. 

2.  Although  there  are  many  tools  to  examine  CBR.N  effects  at  the  engineering  or  systems  level, 
tools  in  the  bio  area  are  generally  lacking.  This  is  especially  true  concerning  some  of  the 
newer,  engineered  biological  threats  and  with  the  retransmission  or  secondary  infections  from 
the  agent. 

3.  Lastly,  it  must  be  recognized  that  there  is  a  lack  of  current  empirical  test  data  available  to  test 
our  models/tools  against.  For  example,  in  the  nuclear  arena,  the  last  open  air  nuclear  test  was 
conducted  in  1962  and  the  test  data  collected  in  that  and  previous  tests  did  not  have  the  fidelity 
that  we  would  want  to  collect  today. 

w'th  regard  to  CONOPS,  the  key  shortfalls  are  highlighted.  While  there  has  been  some  effort  to 
determine  what  capabilities/tasks  are  required  in  response  to  a  catastrophic  event,  there  has  been  no 
real  effort  to  determine  how  these  responses  need  to  be  time-phased.  As  alluded  to  earlier,  the 
majority  of  the  tools  that  support  the  catastrophic  area  are  effects  related  and  are  therefore  related  to 
consequence  management.  There  is  a  distinct  lack  of  effort  concerning  analysis  of  pre-event 
capabilities.  This  may  be  related  to  the  lack  of  appropriate  metrics  in  this  area.  For  example,  how  do 
you  measure  success  with  regard  to  preventing  or  deterring  a  WMD  event? 

With  regard  to  methods  and  tools,  we  ve  already  mentioned  our  concerns  about  how  the  results  from 
one  model  are  not  directly  usable  as  inputs  to  another  model  and  our  concerns  regarding  biological 
event  modeling.  We  also  have  concerns  that  there  are  large  gaps  in  our  tools  to  examine  the  socio- 
economical,  psychological,  and  political  effects  and  that  there  is  really  no  tools  at  the  higher  end  of  the 
M&S  pyramid. 
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•  Data/CONOPS 

♦  DHS  development  of  a  database  of  state  and  local  capabilities  (Target  Capabilities  List) 

♦  Establish  proponents  for  interagency  data  collection  for  engineering  to  campaign  level  analyses 

♦  Developments ssessment  of  appropriate  Analytic  Agenda  scenarios  and  baselines 

♦  Leverage  JFCGM  Experimentation,  Joint  I  raining,  and  Joint  Center  for  Operational  Analysis 
events/processes  for 

Development  of  Combating  WM  Dr  Catastrophic"  analytical  tools/methods 
+  Relevant  data  (e  g  .  Joint  Rapid  Distributed  Database  Development  -  JRD3) 

*  Potential  M&  S/analyli  cal  solutions/need  s  d  erived  through  experimental  i  or 

•  Methods/Tools 

♦  Facilitated  war  games  (EPICS  -  Emergency  Preparedness  Incident  Command  Simulation,  JSRTS 
-  Joint  State  Response  Training  Simulation,  TOPOFF) 

♦  DTRA's  fielding  of  an  integrated  GIG -accessible  suite  of  CBRNE  tools 

♦  Enhancements  to  campaign  models  (ITEM  to  link  IWMDT) 

♦  Commercial  game  adaptalions/gaming  engines  (eg.  SimCity,  Delta3D) 

♦  Potential  partnerships  for  new  M&S/data  development  DTRA,  DHS  {including  Coast  Guard, 

FEMA),  National  Guard  Bureau,  JFCOM,  NORTHCOM.  STRATCOM,  UK,  Canada,  Service  labs 
(AFRL),  industry 


From  the  DoD’s  perspective,  potential  solutions  involve  leveraging  the  expertise  and 
work  to  date  of  other  major  stakeholders  who  can  provide  a  first  cut  at  quantifying  the 
local  response;  DHS  has  the  federal  lead  and  is  the  federal  gatekeeper  in  interactions 
between  federal  and  local/state  planners  and  is  part  way  thru  a  national,  comprehensive 
assessment  of  local  abilities  compared  to  37  Target  Capabilities  (data  is  expected  in  the 
second  half  of  ’06);  the  NGB  has  been  conducting  a  similar  assessment  based  on  State 
Guard  plans  for  selected  National  Planning  Scenarios  (States  with  at  least  one  of  50  most 
populous  cities  had  to  select  NPS  1). 

Interagency  data  is  difficult  to  collect  and  difficult  to  maintain  and  manage.  PA&E/JDS 
can  and  will  manage  data  from  official  scenarios  and  programs  of  record  based  on  the 
analytic  agenda,  but  does  not  manage  unofficial  data.  Identifying  other  proponents  to 
maintain  this  unofficial  set  of  data  will  be  important. 

More  aspects  of  homeland  defense  and  combating  WMD  as  scenarios  in  the  analytical 
agenda  must  be  established  so  more  data  will  be  captured  and  managed  by  PA&E/JDS. 

JFCOM ’s  listed  activities  have  the  potential  to  generate  data  and  possible  CONOPs  to 
support  HLD  analyses  (consequence  management  and  Combating  WMD),  both  as 
examples  of  what  the  DoD  would  do  in  a  response  scenario  and  what  strategies  could  be 
translated  into  options  for  non-DoD  best  practices.  As  JFCOM  is  a  major  customer  for 
DTRA's  simulation  and  analysis  tools,  there  is  an  established  relationship  with  the 
analytic  community  that  could  be  exploited  through  additional  analytical  support  of 
exercises,  wargames  or  data-mining. 

There  are  no  simple  tool  solutions  for  the  analysis  of  catastrophic  scenarios.  To  re¬ 
iterate,  the  WG  agreed  that  the  community  needs  to  look  around  for  new 
tool/modeling/analysis  ideas  and  concepts.  Partnerships  between  the  listed  stakeholders 
also  need  to  be  developed. 
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Focus  on  data  and  establishing  an  M&S  Community  of  Interest 

•  Data 

♦  Establish  a  list  of  questions  for  catastrophic  events  requiring  analysis  (esp  COCOM  needs) 

♦  Collect  exercise  and  operational  data  Unit  commitment,  deployment  timeline,  activities,  duration  of 
commitment 

♦  Collect  war  game  results  C2  relationships,  roles,  CONORS,  metrics,  need  for  DoD  forces.  Title 
10/32  relationships 

♦  Begin  to  revalidate  applicability  Of  basic  engineering  data  (tab  testing  for  CBRNE) 

♦  Develop  Biological  warfare/contagious  disease  data:  Retransmission,  morbidity,  etc 

♦  Follow  up  on  DHS  efforts  to  develop  systematic,  comprehensive  developmenl  of  data  on  local/state 
response  capability 

♦  Support  common  data  standards  (metadata) 

•  Methods 

♦  Increased  catastrophic  scenario  involvement  in  analytical  agenda  products 

♦  Support  the  conduct  of  facilitated  war  games  lo  develop  CONORS,  concepts,  metrics  and  clarity 
responsibilities 

♦  Support  SPG  tasker  to  assess  large  consequence  management  events 

•  Tools 

♦  Assess  DTRA  Collaboration  Center  ability  to  support  analytical  requirements 

♦  Establish  an  interagency  HLD  M&S  COt  to  refine  requirements  and  integrate  effort 


Once  again,  the  WG  summarized  what  it  see  as  the  M&S  needs  into  three  general 
areas  . . .  data,  methods,  and  tools. 

The  first  major  theme  with  regard  to  data  needs  is  the  requirement  to  collect  a  list 
of  questions  from  the  COCOMs  to  identify  what  they  see  as  the  major  analysis 
needs  concerning  catastrophic  events.  The  second  major  theme  is  the  need  to 
collect  data  from  operational  tests,  exercises,  and  wargames  in  order  to  test 
models  being  developed  and  to  serve  as  a  measuring  stick  against  which  we  can 
compare  results  from  current  models.  The  third  major  theme  is  to  follow  up  with 
DHS  efforts  to  develop  data  on  local,  state,  and  other  federal  response 
capabilities.  Lastly,  there  is  a  need  to  develop  common  data  standards  that  can 
facilitate  the  development  of  metadata  in  order  to  standardize  data  collection  in 
the  future. 

With  regard  to  methods,  the  primary  issue  here  is  support  of  the  analytical  agenda 
and  the  expected  tasking  from  the  SPG. 
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•  Specific  to  WG  area 

*  Develop  databases  and  sources  for  (hat  data 

*  Conduct  analytically  supported  exercises,  war  games,  and  studies  lo  refine  CONORS  metrics 

*  Develop  an  active  and  robust  interagency  M&S  Cpmrnu'nity  of  (merest 

*  Consider  redefini  n  g  "Catastrophic"  as  two  rtisst ons  area  s 

■+  Pre-event  (proven! ,  peter.  defeat) 

■4  Post-event  (consequence  management) 

•  Overall  to  Non-Traditional  Challenges 

*  improve  communications  between  all  stakeholders  by  increased  focus  on  Non-traditionat  challenges  in  MORS, 

JCAC.  and  other  joint  and  inter-agency  forums 

4  Develop  M&S  Communities  of  Interest  for  each  of  the  sub-areas 

*  Reapportion  M&S  funding  from  traditional  to  non-traditional  models 

*  Renew  emphasis  on  analytically  supported  war  gaming 

*  Refocus  the  analytic  agenda  on  non -traditional  scenario  development  and  analysis 

*  Data-mme  DTOs.  goals,  objectives,  and  funding  profiles 

•  Other  comments 

4  Need  to  support  inter-agency  participation 

*  Need  to  recognize  mat  significant  policy  and  legal  issues  remain  unresolved 

4  If  the  next  9-1 1  or  OIF  stability  operations  happens,  wN  we  be  able  to  say  we've  done  our  best  to  plan  and  prepare? 


In  our  earlier  slides,  we  noted  that  there  is  generally  a  lack  of  models  and/or  tools 
above  the  engineering  or  systems  level.  Before  development  of  higher  level 
models  can  proceed,  we  need  to  focus  on  developing  good  data  and  sources  for 
that  data.  This  may  be  facilitated  by  the  establishment  of  a  robust  interagency 
community  of  interest  in  the  catastrophic  arena.  We  also  see  the  need  to  redefine 
the  catastrophic  mission  area  into  pre-  and  post-  event  areas.  Current  efforts 
seem  to  be  focused  on  the  post-event  or  consequence  management  actions,  but 
we  cannot  afford  to  forget  analysis  focused  on  preventing  or  deterring 
catastrophic  events. 

In  general  we  see  the  need  for  greatly  increased  interagency  participation  and  that 
there  are  significant  policy  and  legal  issues  that  remain  unresolved.  In  spite  of 
these  shortcomings,  our  efforts  need  to  be  focused  on  improving  our  analytical 
capabilities  in  the  catastrophic  arena.  As  we  face  ourselves  in  the  mirror  each 
day,  our  bottom  line  question  ought  to  be  have  we  done  our  best  to  respond  to  or 
to  prevent  the  next  9- 1 1  event. 
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•  DTRA  develop  paper  on  combating  WMD  for 
PHALANX  to  describe  new  missions  and  required 
supporting  analysis  -  consider  follow  up  meetings 

•  Provide  this  brief  to  Analytical  M&S  Master  Plan 
conference  (March  12-14) 

•  Participate  in  March  MSFD  conference,  Newport, 

Rl  (combating  WMD) 

•  Support  “THOLIAN  WEB” 


Based  on  the  pre-Workshop  WG  3  Survey,  there  seemed  to  be  a  number  of 
“voices  crying  out  in  the  wilderness”  in  terms  of  what  was  going  on  in  DOD  and 
elsewhere  and  what  tools  and  data  are  available  for  analysts’  use.  As  a  result  of 
the  survey,  DTRA  gave  two  briefings  to  educate  the  WG  on  what  DTRA  can 
do/is  doing  and  then  the  tools  that  DTRA  makes  available  for  use.  To  expand 
that  education  to  the  larger  analytic  community,  a  DTRA  representative  agreed  to 
develop  a  paper  for  PHALANX  to  reiterate  what  was  shared  at  Workshop. 

To  provide  needed  priority  for  HLD  activities  and  therefore  get  the  needed 
resource  and  participatory  support  for  analyses,  it  will  be  important  to  pass  the 
conclusions  of  the  WG  to  the  Analytical  M&S  Master  Plan  conference. 

Similarly,  the  dialog  and  conclusions  from  the  WG  need  to  be  inserted  in  the 
March  MSFD  conference  in  Newport,  RI. 

Finally,  as  a  representati  ve  example  of  what  the  WG  recommended  in  its  Ways 
Forward,  the  analytical  exercise,  THOLIAN  WEB,  co-sponsored  by  PA&E/SAC 
and  J8,  needs  the  community’s  support.  This  exercise  will  be  a  first  iteration  in 
PA&E’s  ongoing  attempts  to  define  the  DOD  role  and  required  forces  for  a 
Consequence  Management/HLD  scenario.  THOLIAN  WEB  focuses  specifically 
on  National  Planning  Scenario  #1  (10  kT  IND). 
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Strategic  Deterrence:  The  prevention  of  adversary  aggression  or 
coercion  that  threatens  vital  interests  of  the  United  States  and/or 
our  national  survival.  Strategic  deterrence  convinces  adversaries 
nof  to  take  grievous  courses  of  action  by  means  of  decisive 
influence  over  their  decision  making. 

Ends 

•  Decisively  influence 
Decision  making  calculus 

Ways 

•  Deny  Benefits 

•  Impose  Costs 

•  Induce  Restraint 

Means 

■  Joint  Force  Capabilities 
(DoD) 

■  Elements  of  National 
Power  (USG) 


The  Strategic  Deterrence  Joint  Operating  Concept  (SD  JOC)  provides  the  conceptual  basis  for  this 
working  group.  As  with  other  non -traditional  activities,  deterrence  is  being  redefined  to  more 
adequately  address  a  broader  set  of  conditions  and  adversaries.  The  SD  JOC  presents  a  concept 
that  seeks  to  achieve  decisive  impact  or  influence  over  an  adversary’s  (or  potential  adversary's) 
strategic  decision  making  through  actions  that  are  perceived  by  the  adversary  to  impose  costs, 
deny  benefits,  and  induce  restraint.  As  outlined  in  this  DoD  concept,  strategic  no  longer  means 
nuclear,  and  potential  deterrent  actions  span  the  elements  of  national  power.  The  concept  further 
attempts  to  balance  the  appropriate  use  of  both  carrots  and  sticks  (i.e.,  inducements  as  well  as 
threats)  by  understanding  the  adversary’s  point  of  view. 

During  the  Cold  War,  deterrence  became  the  military  centerpiece  of  a  successful  grand  strategy 
(containment)  aimed  at  countering  Soviet  expansion.  However,  the  opening  years  of  the  2 1  st 
Century  present  many  different  challenges  for  our  military  deterrence  efforts.  Deterrence  must 
now  be  assessed  in  concert  with  the  defense  policy  goals  of  assuring  allies  and  friends,  dissuading 
future  military  competition,  and  decisively  defeating  any  adversary.  These  new  challenges  require 
a  new  concept  for  ‘waging’  deterrence  paired  with  revised  joint  force  capabilities  that  together 
provide  the  President  a  wider  range  of  alternative  military  options  for  deterrence.  Strategic 
deterrence  requires  a  national  deterrence  strategy  that  integrates  and  brings  to  bear  all  national 
resources:  diplomatic,  informational,  military,  and  economic.  (Drawn  from  the  SD  JOC,  February 
2004.) 

The  requirement  to  apply  the  deterrence  concept  to  non-state  actors  adds  additional  challenges: 
“Deterrence  based  only  upon  the  threat  of  retaliation  is  less  likely  to  work  against  leaders  of  rogue 
states  more  willing  to  take  risks,  gambling  with  the  lives  of  their  people,  and  the  wealth  of  their 
nations  .  .  .  Traditional  concepts  of  deterrence  will  not  work  against  a  terrorist  whose  avowed 
tactics  are  wanton  destruction  and  the  targeting  of  innocents;  whose  so  called  soldiers  seek 
martyrdom  in  death  and  whose  most  potent  protection  is  statelessness.”  —  National  Security 
Strategy,  September  2002. 
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♦  How  can  analysts  characterize  and  measure 
uncertainty  concerning  deterrence? 

♦  How  can  deterrence  analysts  determine  and  manage 
second-  and  third-order  effects  in  deterrence 
analysis? 

♦  How  can  analysts  estimate  the  magnitude  of  an 
adversary’s  response  to  a  proposed  deterrent 
action?  Methods  exist  to  determine  the  general 
response  (e.g.,  favorable,  unfavorable).  This 
question  concerns  “How  favorable?”  or  “How 
unfavorable?” 


We  posed  three  questions  to  help  frame  discussions  within  the  working  group,  These 
questions  address  three  persistent  issues  concerning  deterrence  analysis  to  support  the 
concept  on  the  previous  slide. 

1 .  How  can  analysts  characterize  and  measure  uncertainty  concerning  deterrence? 
Uncertainty  comes  in  a  variety  of  forms,  including:  the  uncertain  nature  of  the 
problem,  intelligence  available  to  inform  the  analysis,  understanding  of  the  adversary's 
decision  making  process,  our  ability  to  develop  and  communicate  deterrent  actions,  the 
manner  in  which  deterrent  actions  are  received,  unplanned  actions  that  have 
unintended  consequences,  and  the  dynamic  nature  of  strategic  events.  For  example, 
how  do  we  account  for  learning  by  unidentified  (i.e.,  potential)  adversaries? 

2.  How  can  deterrence  analysts  determine  and  manage  second-  and  third-order  effects 
t«  deterrence  analysis?  While  we  have  methods  to  identify,  and  to  some  extent 
measure,  the  main  effects  (or  intended  effects)  of  our  deterrent  actions  on  adversary 
decision  making,  we  have  difficulties  in  identifying  and  accounting  for  cascading 
effects  that  come  in  a  variety  of  forms.  Some  spillover  effects  are  geographical  (e.g,, 
neighboring,  regional,  global);  some  are  temporal  (e.g.,  short  term  deterrence  may 
contribute  to  longer-term  instability);  while  others  cross  disciplines  (e.g.,  economic 
actions  having  a  diplomatic  effect). 

3.  How  can  analysis  estimate  the  magnitude  of  an  adversary >  's  response  to  a  proposed 
friendly  deterrent  action?  In  most  cases,  we  have  methods  to  determine  the  general 
nature  of  an  adversary’s  response  to  a  specific  deterrent  action  (e.g.,  favorable, 
unfavorable),  What  we  lack  are  methods  to  determine  “how  favorable?”  or  “how 
unfavorable?”  that  response  will  be.  Ultimately  learning  that  reaction  will  help  us 
develop  coherent  packages  of  deterrent  actions. 

"When  viewing  or  analyzing  other  points  of  view,  the  meaning  of  any  communication 
is  the  response  elicited  regardless  of  the  intent  of  the  speaker.”  -  A.  Brown 
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•  Stage  Setting  -  DoD  Strategic  Deterrence  Concept 

■  Deterrence  Assessment  Framework 

•  Problem/Discussion  ■  Effects  in  the  Cognitive  Domain 

Discussion:  Measuring  Uncertainty 

-  Understanding  Chinese  Thinking 

-  Gauging  Adversary  Response 

Discussion;  Adversary  Response 

-  Planning  and  Integrating  Deterrence 

Discussion:  2nd  and  3rd  Order  Effects 

•  Synthesis  -  Problem  Description 

-  Historical  Vignette: 

Deterrence  Analysis  Successes  in  OR 

-  Methods  and  Tool  Availability 

-  Recommendations 


This  chart  depicts  the  structure  we  used  to  achieve  the  workshop  goals  of 
identifying  gaps  in  our  analytic  toolset  as  well  as  take  up  the  three  focus 
questions  on  the  previous  chart.  Each  presentation  served  the  purpose  of 
priming  the  group  for  discussions  on  our  ability  to  analyze  deterrence  issues. 
While  certain  overlaps  exist  among  the  different  presentations  and  discussions, 
the  substantive  effect  of  the  whole  was  more  than  adequate  to  identify  gaps  in 
our  ability  to  analyze  deterrence. 

Following  these  discussions,  the  group  asked  itself  “Have  we  defined  the 
problem  (and  problem  space)  correctly?”  That  is,  we  wanted  to  make  sure  that 
any  effort  to  identify  potential  solutions  (i.e.,  analytic  methods  and  tools) 
should  be  applied  against  the  right  problem. 

In  the  final  break-out  session,  the  working  group  identified  methods  and  tools 
that  have  potential  to  address  certain  portions  of  the  problem,  and  further 
developed  the  set  of  recommendations  contained  in  this  brief. 
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•  Scope  of  deterrence  today 

♦  Ends:  Beyond  the  most  dire  (i.e.,  nuclear)  threats 

♦  Ways:  Beyond  coercion  to  include  inducements 

♦  Means:  Beyond  military  to  include  non-military  (inter-agency) 

•  Specifying  the  problem(s) 

♦  Coercion 

■*  Basic,  extended,  intra-war,  persuasion 

♦  Inducement 

•»  Engagement  (alliances,  coalitions),  cooperation  (international 
organizations),  and  assurance 

•  Language  and  definition 

♦  Deterrence  includes  inducement 

♦  MOEs  and  analysis  for  each  of  these  may  be  different 

♦  Perception  by  others  may  be  different  (assume  deterrence  is  coercion) 


Definitions  are  important:  1)  The  scope  of  deterrence,  as  described  in  the  SD  JOC,  has  expanded;  2) 
The  recent  DoD  specification  of  different  types  of  deterrence  (e.g.,  tailorable,  regional,  global),  may 
demand  different  types  of  analysis,  methodologies,  and  toois,  and,  3)  Lack  of  a  common  language 
(terms  and  definitions)  may  have  an  impact  on  both  analysis  and  execution. 

Scope:  The  scope  of  deterrence  is  broader  now  than  in  the  past,  Ends:  In  the  past,  the  focus  was  on 
most  dire  threats  to  national  survival... usually  nuclear.  Today,  we  face  a  wider  range  of  adversaries 
(e.g.,  non-state  actors,  state  sponsors  of  terrorism)  and  a  wider  range  of  threats  (e.g.,  cyber  attacks, 
terrorism).  Ways:  In  the  past,  deterrence  was  traditionally  defined  as  using  threats  of  denial  or 
retaliation  to  prevent  an  adversary’s  hostile  actions.  Today,  we  include  the  use  of  incentives  to 
improve  the  perceived  consequences  of  inaction  -  a  newer  feature  in  the  academic  debate.  Means: 
Broader  means  include  non-military  as  well  as  military  options. 

Specification:  In  order  to  develop  methodologies  and/or  analytic  tools  to  study  and  assess  deterrence, 
a  number  of  aspects  needed  to  be  specified  further.  Much  as  Patrick  Morgan  had  defined  both  general 
deterrence  and  immediate  deterrence,  the  participants  thought  there  were  at  least  three  levels  of 
deterrence  to  include  intra-war  deterrence.  Also,  academic  definitions  of  deterrence  include:  basic 
deterrence  -  deterring  an  adversary  against  attacking  the  US;  extended  deterrence  -  deterring  attacks 
against  allies;  intra-war  deterrence  -  deterring  escalation  during  a  conflict;  and,  persuasion  -  to  gel  an 
adversary  to  take  an  action  -  such  as  withdrawing  from  seized  territory,  rather  than  preventing  an 
action,  such  as  seizing  the  territory  to  begin  with. 

Language:  The  use  of  language  and  definitions  for  deterrence  could  have  an  impact  on  operational 
and  analytic  considerations.  If  deterrence  is  commonly  thought  to  involve  the  use  of  military  threats, 
deterrence  planning  by  the  US  may  be  misperceived  by  allies  and  adversaries  alike  as  being  purely  or 
primarily  coercive  in  nature,  potentially  resulting  in  misperceptions  of  US  policies  and  unintended 
consequences.  The  term  influence  seems  to  better  describe  the  broader  range  of  actions  involved  in 
deterrence,  however  the  term  strategic  influence  (instead  of  strategic  deterrence)  might  be  problematic. 
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WG  4  -  Stakeholders  and  rn©H5 

nwmy  Opmttuu  ftauuch 

Roles  in  Policy  and  Plans 

•  USG:  NSC,  Inter-Agency,  1CT  Congress 

♦  National  objectives;  National  Security  Strategy* 

♦  Develop  policy 

♦  Prioritization  and  allocation  of  resources 

•  DoD 

♦  Military  objectives:  National  military  strategy 

♦  Develop  policy 

♦  Issue  guidance:  ISC  guidance,  CFG,  SPG 

♦  Prioritization  and  allocation  of  resources 

•  STRATCOM 

♦  Develop  global  deterrence  plan 

♦  Synchronize  deterrence  planning  in  other  plans 

•  Other  COCOMs 

♦  Regional  commands,  including  NORTHCOM:  Develop  regional  deterrence  plans 

♦  SOCOM:  Develop  transnational  deterrence  plans 


Because  the  broader  concept  of  deterrence  is  still  being  developed  in  the  USG, 
it  is  helpful  to  note  stakeholder  roles  in  developing  deterrence  policy  and  plans. 
These  roles  have  implications  for  requirements  and  for  the  execution  of 
deterrence  policy.  Therefore,  this  chart  represents  something  of  a  two-way 
street  with  guidance  for  planning  and  execution  flowing  from  the  top  down  and 
requirements  for  guidance,  resources,  and  collaboration  flowing  from  the 
bottom  up, 

Two  noteworthy  points: 

].  STRATCOM  will  likely  play  a  lynch-pin  role  between  the  policy  and 
guidance  provided  by  DoD  and  the  specific  plans  and  actions  developed  by 
regional  COCOMs.  STRATCOM  is  tasked  both  with  developing  a  Global 
Deterrence  Plan  and  with  synchronizing  specific  deterrence  planning  in 
other  military  plans. 

2.  Regional  COCOMs  will  likely  require  support  from  STRATCOM  to 
execute  their  specific  deterrence  plans.  Likewise,  STRATCOM  may 
require  support  from  the  DoD,  just  as  the  DoD  will  likely  require  support 
from  the  broader  USG  (inter-agency)  to  conduct  deterrence  operations. 
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WG  4  -  Insights  for  the 

|il®35 

Deterrence  Concept 

This  workshop  informed  the  ongoing  revision  of  the  Strategic 

Deterrence  Joint  Operating  Concept  (SD  JOC): 

♦  Definition  of  deterrence 

♦  Scope  of  the  concept 

♦  Cyber-deterrence 

♦  Deterring  non-state  actors 

♦  Communication  of  deterrence  actions 

♦  Planning  and  implementing  deterrent  actions 

♦  Costs  of  deterrence  actions  to  the  US 

- - - - - 

The  timing  of  the  workshop  was  opportune  for  the  further  development  of  the 
deterrence  concept.  USSTRATCOM  is  the  DoD’s  agent  for  maintaining  and 
revising  the  SD  JOC.  In  February  2006,  that  concept  is  mid- way  through  its 
first  revision.  Additionally,  the  working  group  had  the  officer  from 
USSTRATCOM  responsible  for  that  revision  as  a  participant. 

This  list  highlights  some  of  the  more  significant  insights  taken  from  the 
workshop  that  will  feed  directly  into  the  revision  of  the  SD  JOC  (likely  to  be 
re-named  to  the  Global  Deterrence  Joint  Operating  Concept).  Further  details 
on  these  insights  are  given  in  the  back-up  slides. 
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WG  4  -  Attacking  the  Deterrence  Iil635 
Problem  — — — « 


D  i  m  e  n  sic  n  s/f  a  cto  rs 

-Players  (including  capabilities  and 
limitations) 

‘Decisionstections 

-Co  mpetition/coopa  ra  tion 

-  Deterministic  (stochastic/holistic) 

-Goals/objectives  (single  vs  multiple. 

Political,  economic,  military?) 

-Payoffs  (+K  costs/benefrts)  reduce 
problems  of  achieve,  reduce 
bensfiis/increase  costs, 
preference  ordering 

-  Stage  s/pbases 

-Strategies  (risk  averse,  mini-max, 
maxt-min) 

-Inleractions/lnfluences  (2nd,  3rd  order 
effects) 

-Others . 


Match  with  m 

IK 


OR  Methodologies  (Suite) 

-Dynamic  programming  (stochastic  or 
deterministic) 

-Game  Theory  -  (>  2  person),  (non  O-sum), 
(multi-stage) 

-Experimental  gaming 
-Impact/ Influence  diagrams 
-Cost-benefits  analyses 
-Risk  analysis  (p|eventj  *  consequence) 
-Decision  trees/Decision  analysis  (value. 


Application  ot  Methodology  depends  on  the  dimensions  AND  the  question 
being  asked  or  the  decision  to  be  made! 


Appropriate  question:  Do  certain  actions  contribute  (detract  from)  deterrence  or  stability? 
Inappropriate  question:  What  is  the  probability  that  action  X  will  deter  actor  Y  from  doing  Z? 


As  an  initial  synthesis  of  the  group’s  discussion,  this  slide  depicts  the  multi¬ 
dimensional  nature  of  the  problem  on  the  left,  and  some  of  the  more  promising 
operations  research  methods  and  tools  available  to  help  with  deterrence 
analysis  on  the  right.  The  group  immediately  and  unanimously  saw  the  need 
for  a  suite  of  tools  for  even  the  most  basic  deterrence  analysis  problems.  This 
concurs  not  only  with  the  other  working  groups,  but  also  with  the  findings  from 
other  analytic  venues,  in  particular  the  Defense  Adaptive  Red  Team  (D.ART) 
assessment  of  USSTRATCOM’s  deterrence  analysis  capabilities. 

Underlying  this  suite  of  methods  and  tools  is  the  need  for  data,  and  the  nature 
of  that  needed  data  is  changing,  just  as  the  concept  has  changed.  (More  is 
presented  on  the  data  issue  later.) 

The  reminder  at  the  bottom  of  the  slide  serves  to  bring  us  back  to  the 
sometimes-overlooked  imperative  to  define  the  problem  correctly.  In  this  case, 
the  focus  of  a  substantial  part  of  deterrence  analysis  is  to  develop  sets  of 
actions  that  will  serve  to  further  deter  or  to  stabilize.  What  is  often  discussed, 
however,  are  the  difficulties  we  have  in  quantifying  the  probability  of  an 
adversary’s  response.  While  interesting,  that  problem  is  usually  not  the 
effective  focus  of  the  analysis  and  serves  to  distract  the  analysts  and  decision 
makers. 

In  total,  this  slide  represents  an  initial  functional  decomposition  of  the  problem 
and  helped  the  working  group  next  identify  gaps  and  potential  solutions. 
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WG  4  -  Systems  Engineering 
Design  Process 


F71SH5 


Formulation  of 
Alternatives 


Analysis  of 
Alternatives 


Interpretation  of 
Alternatives 


lUMnij 


Source:  USMA  Department  of  Systems  EnBlneering 


In  developing  the  functional  decomposition  on  the  previous  chart,  the  group 
quickly  realized  that  we  were  identifying  methods  and  tools  for  three  broad 
types  of  analytic  problems.  The  first  is  what  we  earlier  referred  to  as  defining 
the  problem  (and  problem  space)  correctly*  The  second  is  analyzing  the 
effects  of  certain  deterrent  actions  on  certain  adversaries*  The  third  is 
analyzing  the  impacts  of  individual  and  multiple  actions  on  individual  and 
multiple  actors,  the  effects  of  all  of  these  actions  over  time,  and  a  variety  of 
other  types  of  second-  and  third -order  effects* 

These  three  problems  roughly  align  with  the  three  steps  of  many  common 
systems  engineering  design  processes.  This  chart  shows  one  such  process, 
from  the  Systems  Engineering  Department  at  the  US  Military  Academy*  It 
provides  a  framework  of  how  to  identify  potential  methods  and  tools,  as  each 
of  these  areas  are  well  established  and  can  be  modified  for  the  deterrence 
effort. 

The  Formulation  of  Alternatives  step  in  the  process  is  concerned  with  defining 
the  problem,  identifying  stakeholders  and  their  needs,  criteria  for  decision 
making  and  developing  sets  of  alternatives*  The  Analysis  of  Alternatives  step 
involves  gaining  a  sufficient  understanding  of  each  alternative  and  its 
relationship  to  the  system  in  order  to  make  a  fully  informed  decision.  The 
Interpretation  of  Alternatives  step  entails  the  framing  of  the  problem  in  an 
understandable  manner  for  the  decision  maker  and  developing  implementation 
plans  to  achieve  the  system  goals* 
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WG  4  -  Methods  and  Tools 

for  Design 

— 

Must  both  Scope 


and  Bound  the  problem!  | 

- - - I 


Effective  Need 
and  Problem 
Understanding 


|  goal  | 


— 

OBJ 

OBJ 

OBJ 

Value  System 
Design 


SDM ,  EPS 
Objectives, 
Criteria ,  and 
Weights 


Needs  Analysis 
Stakeholder  Analysis 
I/O  Modeling 
Pareto  Analysis 
Interaction  Matrices 
Causa!  Loop  Diagrams 
Affinity  Diagrams 
Functional  Decomposition 
Functional  Flow  Diagramming 

Source: 


Value  Models  (Qual  &  Quant} 

Gold,  Silver,  &  Platinum  Standards 
Affinity  Diagrams 
Local  Weights 
Global  Weights 

System  Definition  Matrix  (SDM) 
Eng  Problem  Statement  (EPS} 

USMA  Department  of  Systems  Engineering 


Brain  Storming 
Brain  Writing 
Dynamic 
Confrontation 
Zwicky's 

Morphological  Box 

Feasibility 

Screening 


As  mentioned  on  the  previous  slide,  one  advantage  of  looking  at  an 
engineering  system  design  process  is  that  it  allows  us  to  draw  upon  a  well- 
developed  field  of  expertise.  For  example,  this  slide  depicts  general  methods 
and  tools  to  help  system  analysts  accomplish  each  of  the  sub-steps  in  the 
Formulation  of  Alternatives  portion  of  the  process.  Similar  lists,  as  well  as 
descriptions  of  each  method  or  tool,  are  available  for  the  remaining  portions  of 
the  process. 

The  caution  at  the  top  of  the  chart  reminds  us  that  we  need  to  first  consider  all 
possible  aspects  of  the  problem  (scope),  and  then  eliminate  irrelevant  or 
infeasible  aspects  of  the  problem  (bound)  so  that  we  are  left  with  a  concise 
definition  of  the  problem  that,  when  solved,  will  address  the  effective  need. 

Insufficient  understanding  and  scoping  of  the  problem  often  leads  to  solutions 
that  do  not  account  for  the  significance  in  the  relationships.  In  this  case, 
insufficient  scoping  would  likely  cause  critical  2nd  and  3rd  order  effects  of 
deterrent  actions  to  be  omitted  or  misused,  and  significant  or  important  sources 
of  uncertainty  to  be  overlooked.  On  the  other  hand,  a  feeble  bounding  effort 
can  often  lead  to  problems  that  are  intractable,  either  because  of  their 
complexity  or  because  the  data  requirements  are  infeasible.  In  this  case,  not 
bounding  the  deterrence  problem  (e.g.,  trying  to  analyze  general  deterrence 
that  accounts  for  all  global  effects  and  interactions)  would  only  bog  down  the 
objective,  and  most  likely  expend  the  analytic  budget  more  rapidly. 
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Combining  the  functional  decomposition  presented  earlier  with  the 
understanding  of  the  system  engineering  design  process  gives  us  a  framework 
to  assess  method  and  tool  availability  for  deterrence  analysis.  This  is  certainly 
not  a  definitive  mapping  of  tools  to  problems!  In  fact,  even  if  we  could 
account  for  all  significant  methods  and  tools,  the  mapping  would  be  incomplete 
as  the  potential  changes  based  on  the  nature  of  the  problem.  This  means  that 
what  is  learned  in  the  problem  definition  step  in  part  determines  which  tools 
are  appropriate  for  analysis  later  in  the  process. 

Instead,  this  chart  serves  to  indicate  a  general  potential,  using  the  professional 
estimate  of  the  workshop  participants.  In  some  cases,  these  methods  and  tools 
have  already  been  applied  to  the  indicated  analytic  tasks;  in  other  cases,  they 
have  not.  More  importantly,  the  group  identified  a  number  of  tools  that 
theoretically  could  be  applied  to  certain  tasks,  but  whose  data  collection  would 
likely  drain  the  national  budget,  or  are  otherwise  completely  infeasible.  For 
example,  mathematical  network  models  used  to  examine  complex  systems 
might  help  one  to  understand  the  decision  making  processes  in  terrorist 
networks,  but  that  understanding  would  likely  require  describing  each  node  and 
edge  in  the  network  to  some  sufficient  level. 

In  his  plenary  brief,  Dr.  Hanley  reminded  the  workshop  that  analytic  tools  must 
be  usable  (interpretable),  extendable  and  replicable.  To  these  criteria,  OSD's 
COL  Patrick  Kelly  added  the  reminder  that  the  analysis  must  be  completed  in 
time  to  be  relevant  to  the  decision  maker. 
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WG  4  -  Method  and  Tool  Availability  msHs 
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♦  Significant  Gaps  in  determining  the  impacts  of 
second  and  third  order  effects,  and  measuring  the 
magnitude  of  adversary  responses. 

♦  Data  availability  eliminates  some  models. 

♦  Timely  methods  required  to  inform  decisions. 

♦  Human-in-the-ioop  methods  often  best  choice. 
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While  the  method  and  tool  mapping  highlights  general  potential,  it  does  not 
capture  specific  strengths  and  weaknesses,  which  specific  tools  are  best  suited 
for  specific  problems,  or  what  priorities  among  tools  and  gaps.  However, 
based  on  the  expertise  of  the  group,  we  were  able  to  draw  some  general 
conclusions: 

•  As  expected,  there  are  no  silver  bullets  waiting  to  be  discovered  that  will 
address  all,  or  even  most,  of  the  aspects  of  the  deterrence  analysis 
problems. 

•  We  face  significant  gaps  in  two  of  the  three  focus  areas  laid  out  at  the 
beginning:  determining  the  impacts  of  second  and  third  order  effects,  and 
determining  the  magnitude  of  an  adversary’s  response.  The  focus  area  of 
characterizing,  reducing  and  managing  uncertainty  is  substantially  covered 
with  the  well  developed  risk  analysis  methods. 

■  As  mentioned  previously,  the  data  requirements  for  some  approaches  likely 
rule  out  those  approaches. 

•  Because  of  the  data  non-availability  problem,  the  human-in-the-loop 
(HITL)  approaches  (e.g.,  role  playing,  wargamming)  are  often  the  best 
choice.  This  is  completely  reasonable  given  the  current  state  of  the  art  for 
deterrence  analysis.  HFTL  methods  are  often  excellent,  and  inexpensive, 
choices  for  addressing  formative  problems. 
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WG  4  -  Additional  Points  to  m@R5 
Consider  1 

*  Problem  Definition 

♦  The  problem  is  well-scoped;  work  is  needed  to  bound  the  problem  (all  deterrence, 
or  just  strategic  deterrence?  All  forms  of  influence?) 

♦  Throughout  the  workshop,  we  used  at  least  seven  deterrence  terms 
1*  Strategic  deterrence 

2.  Tailored  deterrence 

3.  Global  deterrence 

4.  Transnational  deterrence 

5.  Regional  deterrence 

6.  Immediate  deterrence 

7.  General  deterrence 

♦  How  do  we  account  for  the  dynamic  nature  of  the  problem? 

*  Metrics 

♦  Are  we  asking  the  right  questions  to  guide  metrics  development?  (Instead  of 
trying  to  respond  to  the  question,  "What  are  the  metrics?") 

♦  What  are  those  questions? 

♦  What  decisions  will  be  informed  by  the  metrics? 


Here,  well -scoped  indicates  that  a  sufficient  number  of  important  relationships 
concerning  strategic  decision  making  have  been  identified.  This  does  not  mean 
that  these  relationships  have  been  described  in  detail  or  adequately  modeled, 
Streamlining  the  problem  (essentially,  determining  the  context  of  interest)  is 
more  difficult  to  do,  particularly  in  the  abstract.  However,  at  the  top  level, 
some  streamlining  would  enhance  deterrence  analysis  simply  by  providing 
focus.  For  example,  of  the  seven  deterrence  terms  on  the  chart,  only  the  first 
has  a  clear  definition  adopted  by  the  DoD. 

The  dynamic  nature  of  the  deterrence  analysis  problem  becomes  more 
important  as  the  DoD  continues  to  move  tow'ard  a  continuous  and  global 
approach  to  deterrence.  Essentially,  this  means  that  future  deterrence  plans 
will  not  be  static  plans  but  will  be  plans  that  will  be  continuously  reviewed  for 
execution,  with  feedback,  revisions,  and  constant  consideration  of  new  possible 
courses  of  action. 

These  problems  are  dynamic  in  that  they  change,  even  as  they  are  being  solved. 
Not  only  do  the  elements  of  the  problem  change  over  time,  but  fundamental 
relationships  change  over  time. 

Useful  metrics  generally  flow  from  a  well-defined  problem.  If  analysts  start  by 
defining  metrics  (before  defining  the  problem),  the  resulting  set  of  measures  is 
often  inappropriate  or  unusable, 
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WG  4  -  Additional  Points  to  lilSffS 
Consider  2 

•  Methods 

♦  Allow  for  competitive  analysis  to  provide  better  integration  of 
knowledge  and  methods;  not  all  redundancy  is  bad 

♦  There  is  value  in  outsourcing  analysis;  can  help  avoid  single  points 
of  failure 

♦  Consider  the  impact  of  expert  opinion  as  a  source 

♦  Analyses  requires  inter-disciplinary  collaboration 

♦  Develop  methods  to  validate  analysis 

♦  Develop  methods  to  address  the  role  of  ambiguity  in  deterrence 

♦  Methods  must  be  usable/understandable,  extendable,  and 
replicable 


In  our  quest  to  use  our  resources  efficiently,  we  often  try  to  remove  all 
redundancies.  However,  in  many  cases  having  different  groups  work  on  the 
same  problem  is  very  helpful  in  providing  insight  into  the  nature  of  the 
problem,  especially  regarding  the  uncertainties  involved. 

In  general,  methods  exist  for  many  aspects  of  deterrence  analysis,  but  new  or 
refined  methods  are  required  for  some  of  the  more  difficult  parts.  Because  the 
object  of  the  analysis  (the  adversary’s  decision  making  calculus)  is  very 
difficult  to  observe,  measure,  and  analyze,  it  is  not  surprising  that  methods  that 
deal  with  the  uncertain  nature  of  the  problem  are  at  the  top  of  the  list  of  needed 
tools. 
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WG  4  -  Recommendations  li1©R5 

Hfluw  Otmtinnv  bwnh  SOfkHy 

•  To  DoD/Sponsors 

♦  Carefully  define  scope  and  roles  concerning  deterrence 

♦  Drive  development  of  deterrence  policy,  analysis  and  planning  across  USG 
and  DoD 

♦  Sponsor  focused  research  in  difficult  areas,  e.g,  integration  of  analysis, 
characterization  of  uncertainty 

•  To  MORS 

♦  Provide  a  mentor  to  each  WG  (a  la  Rist  mentors) 

♦  Engage  MORS  to  accommodate/encourage  greater  IA  participation 

♦  Hold  MORS  Workshop  on  Influence  Operations 

♦  Sponsor  MORS  competition  for  difficult  problems 


We  look  to  the  DoD  to  provide  the  needed  definitions,  guidance,  and  policy 
concerning  deterrence.  These  elements  would  serve  to  not  only  define  roles 
and  responsibilities,  but  they  would  also  drive  deterrence  analysis  and  planning 
across  the  DoD  (and  USG).  By  sponsoring  research  along  the  lines  of  the 
difficult  analytic  tasks  in  this  brief,  the  DoD  would  help  energize  and 
synchronize  the  greater  community  of  interest. 

MORS  is  uniquely  positioned  to  play  a  critical  role  in  continuing  the 
development  of  OR  tools  and  methods,  and  more  importantly,  the  development 
of  analysts  to  tackle  these  tough  problems.  In  particular,  broadening  the 
traditional  scope  of  MORS  beyond  the  DoD  will  help  to  move  the  analytic 
community  closer  to  the  problem,  help  define  boundaries,  and  better  prepare 
analysts  for  the  tough  work  ahead. 
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WG  4  -  Backup  Material  1  fil6H5 

MUttwv  OparaOon* 


Workshop  Insights  to  inform  the  Strategic  Deterrence 
Joint  Operations  Concept 
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WG  4  -  Deterrence  Concept  m©H5 
Insights  1 

Definition  and  Scope 

•  Definition  -  reconsider  classic  definition 

♦  Classic  deterrence  primarily  considers  cost/benefit  of  action 

♦  Also  consider  -  Cost/benefit  of  inaction  (SD  JOC  -  induce  restraint) 

•*  Not  a  classic  definition  of  deterrence,  but  practical 
-»  Includes  inducement  for  inaction  or  alternative 
■*  Deterrence  is  necessary,  but  not  sufficient  for  stability 
-*■  Desire  cost/benefit  of  inaction  is  greater  than  the  cost/benefit  of  action 
Achieve  stability  within  deterrence 

•  Current  scope  of  JOC  is  Strategic  Deterrence 

♦  Expand  discussion  that  concept  may  be  applied  at  a  strategic,  operational, 
or  tactical  level  to  achieve  a  strategic  effect 
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WG  4  -  Deterrence  Concept 
Insights  2 


m&?i5 

Hllhjrr  DpoilMti  i 


Application 

•  Strategic  concept  can  also  be  applied  by  operational  and  tactical  level 
commanders  to  deter  attack 

♦  Requires  some  level  of  adversary  analysis 

m  Discuss  cyber-deterrence  as  another  deterrence  objective 

•  Manage  expectations  of  deterrence 

♦  Canl  deter  all  attacks  4  credibility  will  suffer 

♦  If  the  attack  is  not  catastrophic,  then  we  mitigate  effects  and  drive  on 

♦  Attacks  will  happen  in  a  long  war 

♦  Broad  policy  statements  are  potentially  difficult  in  application 

•  Expand  the  discussion  of  Immediate  versus  general  deterrence 

♦  Go  ns  id  er  effects  of  immediate  deterrence  actions  on  general  deterrence 

•  Examine  the  efficacy  of  clarity  of  intent  (including  demonstrated 
capability)  versus  declaratory  policy 
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WG  4  -  Deterrence  Concept  IilSS5 
Insights  3 

Communicating  Deterrent  Actions 

•  Focus  on  the  how  of  communication  as  well  as  the  what 

♦  The  meaning  of  any  communication  is  the  response  elicited,  regardless  of  the  intent 

•  Communication  with  adversary  -  learn  how  to  effectively  communicate 

♦  Deiivery  Methods  -  What  sources  does  adversary  pay  attention  to?  What  sources 
does  adversary  trust? 

♦  Format  of  the  message  -  to  impact  or  to  influence? 

♦  How  do  we  effectively  communicate  with  adversary? 

•  Examine  how  messages  will  be  delivered  and  manipulated 

•  Is  there  a  model  that  can  help  demonstrate  how  communication  of  a 
message  occurs 

♦  Primary  effect  on  adversary 

♦  Secondary  effect  on  other  parties,  domestic  audience 

♦  Is  there  a  bio-model  that  applies? 

♦  Can  we  inoculate  certain  societies  against  misinformation? 
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WG  4  -  Deterrence  Concept  ni@35 
Insights  4  — — — , 

Non-State  Actors 

•  Examine  the  efficacy  of  deterring  terrorists 

♦  May  not  be  able  to  expend  the  necessary  resources  and  time  to  effectively  deter 

♦  US  may  be  morally  restrained  from  imposing  costs  (or  providing  benefits) 

♦  Benefit  denial  may  be  of  increasing  importance  with  terrorists 

•  Difficult  to  identify  (and  thus  understand)  the  actor  to  deter 

•  Often  not  interested  in  stability  -  cost  of  inaction  may  be  too  high 

•  Want  to  upset  the  status  quo  -  for  purpose  of  changing  the  system 

•  Can  we  deter  those  committed  to  dying  for  a  cause?  Yes,  decision 
calculus  still  applies 

•  Costs  of  deterring  extremists  may  be  too  high 

■  In  many  cases  deterrence  has  already  failed;  further  deterrence  efforts  are 
escalation  control 

•  Denying  benefits  often  most  effective  way  to  deter  terrorists 

•  Need  to  determine  how  to  influence/deter  a  networked  actor  (model 
network) 
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WG  4  -  Deterrence  Concept  IilS35 
Insights  5  — 

Planning  and  implementation 

•  Require  the  ability  to  characterize  uncertainty  in  decision  models 

♦  Uncertainty  rises  from  a  lack  of  understanding  of  adversary  decision  making 

♦  Ability  to  model  uncertainty  would  be  very  helpful  to  decision  makers 

♦  Determine  how  to  design  strategy  and  plans  around  a  range  of  uncertainty 

♦  Eliminate  unlikely  cases,  incorporate  cases  with  dramatic  impact 

•  Require  the  ability  to  model  non-DoD  means  that  affect  deterrence 

*  Require  models  to  show  deterrent  effects  caused  by  proposed  changes  in  force 
posture/basing 

*  Require  models  that  demonstrate  deterrent  impact  of  nuclear  weapons  on 
deterrence  (more  than  a  mathematical  model  of  damage) 

•  Need  tools  to  help  show  2nd  and  3rd  order  effects  of  deterrent  actions 

♦  Effect  on  who  you  are  targeting? 

♦  Effect  on  other  adversaries,  allies,  friends,  neutrals  (3rd  party) 

*  Need  models  to  gain  insight  into  various  methods  to  deter  non-state  actors 
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WG  4  -  Backup  Material  2  li1@H5 
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Systems  Engineering  Design  to  Help  Frame  the 
Deterrence  Analysis  Problem 
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WG  4  -  Systems  Engineering  Design  Process  mefts 

mnrnrr  Ovnbm  facMrtv 


Formulation  of 
Alternatives 


Analysis  of 
Alternatives 


Interpretation  of 
Alternatives 


Major  Steps 


Feed  Forward 


Feedback 
&  Iterate 


Source:  USMA  Department  of  Systems  Engineering 
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WG  4  -  Systems  Processes  Mapping  1 


li1©R5 
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Systems  Engineering  Design 
Process  { SEDP ) 

Systems  Engineering  end 
Management  Process  (SEMP) 

Application:  Military  Decision 
Making  Process  (MDMP) 

1.  Formulation  of  alternatives 

a.  Needs  analysis 

b.  Value  systems  design 

c.  Synthesis  of  alternatives 

1.  Problem  definition  phase 

a.  Needs  en  a  lysis 

b.  Value  systems  design 

2.  Design  and  analysis  phase 

a.  Alternatives  generation 

2.  Analysis  of  alternative* 

a.  Modeling  and  analysis 

b.  Refinement  of 

alternatives 

b.  Modeling  and  analysis 

3.  Interpretation  of  alternatives 

a.  Decision  making 

b.  Planning  for  action 

r 

. 

3.  Decision  making  phase 

a.  Arte  motives  scoring 

b,  Decision 

4.  Implementation  phase 

a.  Planning  for  action 

b.  Execution 

c.  Assessment  and  aontro! 

6.  Rehearsal 

9,  Execution  and  assessment 
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WG  4  -  Deterrence  Assessment  Process  Fi1©H5 

MIIIVY  OpntiwK  fcM.rm  Wmy 
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WG  4  -  Systems  Processes  Mapping  2 


lil©H5 
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Sys/ems  Engineering  Design  Process 
(SEDP) 

Application:  Strategic  Deterrence  Ana/yste 

1 .  Formulation  of  alternatives 

a  Needs  analysis 

b.  Value  systems  design 

c.  Synthesis  of  alternatives 

2.  Analysis  of  alternatives 

a  Systems  modeling  and  analysis 

b.  Refinement  of  alternatives 

5§§§| 

3i~zr 
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Analytic  Mapping 
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•  Needs  analysis 
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•  Interaction  matrices 
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■  Functional 

decomposition 

•  Value  models  (qualitative 
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•  Affinity  diagrams 

•  Goal  weighting 

•  System  definition  matrix 
(SDM) 

•  sssar- 

.  Zwicky's  Morphological  Box  j 
•  Feasibility  screening 
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Analytic  Mapping 
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•  Dynamic  programming 

•  Cost-benefit  analyses 

•  Risk  analysis 

•  Decision  analysts 

•  Sensitivity  analysis 
■  Recommendation 

•  Presentation 

•  Project  management  techniques 
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I718H5 

NUMarr  fipnan  Iwth  faKMtfv 

Analysis  for  Non-Traditional 
Security  Challenges:  Methods  & 

Tools 

Military  Operations  Research  Society 
(MORS)  Workshop 

WG  5  -  Decision  Making  And  integration 

Outbrief 

Dr.  Greg  Parnell,  USMA 
Steve  Richter,  JHU/APL 


This  document  provides  an  annotated  briefing  of  the  report  of  Working  Group  5,  Decision 
Making  and  Integration. 
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WG  5  -  Charter  msRs 

Httnary  Dpntton*  SocWr 

•  Initial 

♦  “Effective  decision  making  requires  new  methods  and  tools  to  incorporate 
non-traditional  challenges.  Existing  methods  of  collaboration  will  need  to  be 
reevaluated  to  consider  the  effects  of  incorporating  the  new  challenges 
across  all  levels  of  classification.” 

•  Final 

♦  “Using  the  QDR  as  a  starting  point,  assess  the  decision  making  and 
integration  issues  of  future  non-traditional  security  challenges.  Focus  on 
force  sizing  issues  for  FY08  POM.’ 

WG-5  Contribution 

We  assessed  our  ability  to  measure  the  capabilities  identified  in 
the  four  QDR  Focus  Areas.  We  concluded  that  we  need  better 
measures  for  many  of  the  capabilities  that  will  have  the  most 
impact  on  non-traditional  security  challenges.  We  also  found  that 
risk  analysis  may  be  the  most  promising  resource  allocation 
framework  for  two  of  the  four  areas. 


The  co-chairmen  of  the  working  group  struggled  with  the  definition  of  “decision  making  and 
integration”  as  applied  to  this  workshop.  Numerous  discussions  with  workshop  sponsors 
resulted  in  a  refined  charter,  one  more  focused  on  assessing  the  implications  of  QDR  2005. 
The  group  was  tasked  with  examining  the  capabilities  the  QDR  identified  as  critical  for 
meeting  the  strategic,  operational  and  tactical  objectives  of  four  focus  areas: 

1 .  Defeating  terror  networks; 

2.  Defending  the  homeland  in  depth; 

3.  Shaping  choices  of  countries  at  strategic  crossroads;  and, 

4.  Preventing  the  acquisition  or  use  of  WMD  - 

and  then  assessing  the  implications  of  those  capabilities  with  respect  to  impact  on  force  size 
and  shape. 

Working  Group  5  found  that  deficiencies  lie  in  our  ability  to  measure  how  well  these 
capabilities  answer  non-traditional  security  challenges,  and  that  these  shortfalls  will  need  to 
be  mitigated  in  order  to  better  understand  resource  allocation  requirements  —  not  only  within 
the  DoD,  but  across  all  government  agencies  tasked  with  supporting  response  to  non- 
traditional  security  challenges  we  will  face  during  the  long  war. 

Finally,  Working  Group  5  presented  some  analysis  that  indicates  that  the  nature  of  these 
challenges  may  call  for  a  broadening  of  our  analytic  tool-kit,  to  include  wider  use  of  social 
science  and  risk  analysis  methodologies,  particularly  in  the  areas  of  WMD  and  defense  of  the 
homeland. 
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WG  5  -  Overview  li1@R5 

>1  Op— HOIM  flBiuith 

•  WG  5  framework  for  analysis 

•  Characterize  the  future  (2025) 

♦  Environment 

♦  Threats 

♦  Opportunities 

•  Focus  on  decision  making 

♦  Decision  makers 

♦  Decision  maker  questions 

♦  Integration  challenges 

•  Assess  QDR  capabilities 

♦  identify  capabilities  that  will  influence  force  sizing 

♦  Identify  or  develop  measures  for  force  sizing  capabilities 

•  Assess  resource  allocation  frameworks 

•  Recommendations 

•  Next  steps 


This  slide  provides  an  outline  of  our  WG  brief. 
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WG  5  -  Decision  Making  and 


Integration  mess 


Irregular  Challenge 


Traditional  Challenges 


Catastrophic  Gvnlhmges 


Disruptive  Challenges 


Ai  iIm>  shows*  rhe  Ei<*ptirimiim  is  shilling 

its  portfolio  of  capabilities  id  itddrtfs*  insular. 
c^Urtroprhicand  diwuptiri*  challflfwi^  whj  ^sustain  in£ 
/,Tpahil«i»*s  to  ddiirass  traditional  ^halien-es.. 


QDR.  page  19 


*  Framework  for  Analysis  for  each  of 
4  QDR  focus  areas 

+  Characterize  future  (2025) 

4  Range  of  operations  that  our 
forces  may  be  required  to 
support 

♦  Identity  decision  making  and 
integration  questions 

♦  Assess  QDR  capabilities 

;  *  identify  capabilities  that  will 

influence  force  sizing 

■*  Identify  or/develop  measures 
for  force  sizing  capabilities 
■  How  would  this  change  the  joint 
analytic  agenda 

♦  Time,  phase 


In  order  to  provide  context  for  follow-on  analysis,  Working  Group  5  thought  it  necessary  to 
discuss  and  describe  the  future  environment,  including  potential  changes  to  the  threat, 
strategic  drivers  and  advances  in  technology.  Under  this  redefined  future,  considerations  for 
future  force  employment  were  discussed,  and  then  mapped  to  the  set  of  required  capabilities 
listed  in  the  2005  QDR  report. 

The  group  focused  on  decision  making  and  integration  challenges  presented  by  the  future 
environment  in  the  context  of  future  force  planning  under  the  QDR  2005  focus  areas.  The 
basic  questions  the  group  tried  to  answer  were: 

“Given  the  four  focus  areas  within  the  non-traditional  security  challenges,  which 
capabilities  will  influence  force  size?” 

and 

“How  can  we  measure  those  capabilities?” 
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WG  5  -  QDR  Focus  Area  Team  Members 

MU"", 

SocMty 

Defeat  Terror 

Defend 

Shaping  Chorees  of 

Preventing  the 

Networks 

Homeland  in 

Countries  at  Strategic 

Acquisition  or 

Depth 

Crossroads 

Use  ofWMD 

CDR  Bryan  Clark 

Chris  Appleby 

Jeff  Brown 

Venton  Duncan 

Amy  Henninger* 

Sue  Iwanski,  FS* 

Jason  Dechant* 

Bill  Greer 

Steve  Hunt 

Alice  Ling 

Kim  DeWitt 

Erik  Jilson 

LTC  Zvl  Mintzer 

Bryan  Unkous 

Craig  Munshower 

Steve  Richter 

John  Regner 

Rafael  Matos 

Janet  Peasant 

Phil  Sauer* 

Wayne  Zandbergen 

CART  John 

Marla  Roth 

Yurchak 

Tom  Sterle 

'  Lead 

Each  focus  area  team  produced  annotated  briefing  of  their  analysis. 

Working  Group  5  was  divided  into  four  sub-groups  —  one  for  each  of  the  four  QDR  focus 
areas.  Each  group  (team)  was  provided  the  same  task,  and  asked  to  proceed  independently. 
The  objective  was  to  look  at  each  focus  area  and  integrate  the  results,  identifying  common 
elements  of  the  assessments  of  future  threats,  the  strategic  environment  and  opportunities. 

The  teams  were  asked  to  identify  key  decision  makers  —  and  postulate  the  type  of  questions 
they  might  ask  in  each  of  the  four  focus  areas. 

Each  team  produced  an  annotated  briefing  of  their  analysis. 
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WG  5  -  Future:  Environment  lilS^S 

MHftPK  Cwitkra  KhwA  HkWt 

•  Defeat  terrorist  network 

♦  The  internet  wilt  be  pervasive  and  exploitable  for  good  and  evil 

♦  Data  will  be  plentiful  but  noisy  and  hard  to  analyze  for  desirability 

•  Defend  homeland  in  depth 

♦  Status  quo  or  significant  shift  in  magnitude  of  defense  budget  and/or  authorities 

♦  l n te grate d  or  b a  1  ka n ized  interage ncy  operati o n s 

♦  High  trust  partnerships  or  low  trust,  independent  relationships 

•  Shaping  choices  of  countries  at  strategic  crossroads 

♦  Complexity  created  by  influencing  factors  (e.g,.  foreign  governments,  NGOs. 
industry) 

♦  Capability  deltas  between  allies,  partners,  and  adversaries 

•  Preventing  the  acquisition  or  use  of  WMD 

♦  Ad  ve  rsa  hes  -  s  ta  te ,  non-state ,  indivi  d  u  a  I  s  wi  th  i  n  creas  ed  a  ccess  to 
k  n  o  wte  dg  e/ resou  rces 

♦  Cyber  and  near  space  concerns 

Future  catastrophic  events  will  shape  the  environment 


In  describing  the  future,  the  groups  concluded  that  the  proliferation  of  data  and  information, 
coupled  with  the  ability  to  rapidly  share  that  information,  creates  a  level  playing  field  for  the 
US,  her  partners  and  adversaries.  While  data  is  expected  to  be  plentiful,  the  challenge  will 
continue  to  be  in  sifting  the  information  to  find  elements  that  contribute  to  problem 
description,  analysis  and  solution. 

Uncertainties  in  the  expected  degree  of  government  agency  cooperation,  future  levels  of 
funding  and  the  evolution  of  partnerships  (both  within  the  structure  of  government  and 
internationally)  present  challenges  for  effective  defense  of  the  homeland. 

The  future  environment  will  be  complex  and  driven  by  numerous  factors.  Shaping  objectives 
will  be  complicated  by  interests  that  extend  beyond  traditional  government  relationships,  and 
will  be  influenced  by  differing  levels  of  technological  capability,  operational  capabilities  and 
system  interoperability  amongst  cooperating  military  forces,  law  enforcement  and  other 
agencies. 

Future  catastrophic  events  will  shape  US  policy,  decisions  and  courses  of  action. 
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WG  5  -  Future:  Threats  in  Long  War  IilSH5 


*  Terrorist  organizations  and  individuals 

♦  State  sponsored 

♦  Ideological  (not  state  sponsored) 

♦  Religious  fundamentalists 

♦  Anti -globalize  Won  groups 

♦  Criminally-based  networks 

*  Strategic  drivers 

♦  Geo-politics  will  shift  QDR  2005  key  regions 

♦  Globalization  and  proliferation  of  technology 

♦  Impact  of  demographic  trends 

♦  Competition  for  strategic  resources  (energy,  water,  eta) 

*  Threat  technology  advances 

♦  CBRNE/future  technology  (nano,  biological);  novel  technologies  outside  WMD 
paradigm 

♦  Indiscriminate  adversary  delivery 

♦  New  combinations 

♦  Perception  control  and  management 


Asymmetric  challenge  -  cost  imposing  strategies 


Many  expect  terrorism  to  be  the  dominant  threat  in  the  long  war*  However*  beyond  terrorism 
fueled  by  religious  or  ideological  extremism,  anti -globalization  groups  may  become  more 
active  (and  perhaps,  more  violent)  in  attempts  to  disrupt  the  global  economy  by  harassing  the 
flow  of  energy  and  transportation  of  goods  and  services. 

A  challenge  for  the  US  and  her  partners  will  be  in  predicting  future  geographic  hotspots.  The 
2005  QDR  has  indicated  a  shift  in  US  strategic  thinking  from  the  long-standing  three-hub 
construct  to  one  that  takes  a  more  nation- specific  approach*  It  is  possible  that  other 
challenges  (regional  and  trans-national  instead  of  national)  will  become  evident,  perhaps 
rendering  current  plans  and  programs  less  than  optimal.  Inversely  related  demographic  trends 
in  the  developed  and  developing  worlds  may  be  an  important  indicator  as  we  attempt  to 
remain  appropriately  focused  upon  future  security  challenges*  Religious  extremism  may  be 
replaced  by  competition  for  scarce  resources  as  the  key  strategic  driver  of  world  events. 

The  long  war  is  expected  to  be  asymmetric*  America  may  be  faced  with  countering  cost- 
imposing  strategies  executed  by  non-state  actors* 
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WG  5  -  Future:  Opportunities  msR5 
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•  For  all  four  focus  areas  there  are  significant 
opportunities  for  cooperation  at  multiple 
levels 

♦  Interdisciplinary 

♦  Intraservice 

♦  Intradepartmental 

♦  Interagency 

♦  Regional 

♦  International 


All  four  subgroups  identified  opportunities  for  (and  perhaps,  the  necessity  of)  increased 
cooperation,  collaboration  and  integration  across  organizational  boundaries.  The  groups  felt 
that  the  nature  of  the  non-traditional  security  challenge  will  require  cooperation  and 
integration  —  not  only  organizationally,  but  ffom  force  planning,  resource  allocation  and 
operational  perspectives  as  well. 

We  believe  that  the  first  step  for  military  operations  research  analysts  is  increased 
multidisciplinary  participation  in  our  studies  and  analyses.  We  will  need  to  include 
psychologists,  sociologists,  economists,  international  relations  experts,  and  many  other 
disciplines. 
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WG  5  -  We  Need  to  Identify  the 
Decision  Makers 


171 6R5 
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Decision  makers  and  stakeholders  span  Defense  Senior  Leader  Council, 

Senior  Leader  Review  Group,  OSD,  Services,  government 
_ departments,  agencies,  Congress,  allies  and  partners. 


•  Defeat  terrorist  network 

♦  SOCOM 

♦  Other  COCOMs 

•  Defend  homeland  in  depth 

♦  NORTHCOM,  DHS 

♦  US  State  Governors  and  National  Guard 

•  Shaping  choices  of  countries  at  strategic  crossroads 

♦  SECDEF,  CJCS,  COCOMs 

♦  Interagency 

•  Preventing  the  acquisition  or  use  of  WMD 

♦  STRATCOM,  NORTHCOM,  DHS 

♦  US  State  Governors  and  National  Guard 


This  slide  identifies  some  of  the  major  decision  makers  and  stakeholders  involved  in  the  non- 
traditional  security  challenges.  In  addition,  each  focus  area  has  a  complex  set  of  execution 
authorities. 

The  decision  making  for  traditional  security  challenges  is  complex.  By  contrast,  the 
subgroup  teams  found  that  the  structure  required  to  ensure  integrated,  coordinated  decisions 
in  support  of  the  non-traditional  challenge  to  be  exponentially  more  complicated.  This 
increased  complexity  results  from  the  need  for  intraservice,  interservice,  interdepartmental 
and  interagency  collaboration  and  cooperation. 
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WG  5  -  Expected  Questions  From 
Decision  Makers 

•  Defeat  terrorist  network 

♦  What  is  the  return  on  investment  within  capabilities? 

♦  What  are  the  trade  spaces  between  capabilities? 

•  Defend  homeland  in  depth 

♦  What  is  the  relative  payoff  of  awareness  and  consequence  management? 

♦  What  balance  of  capabilities  across  layers  of  defense  (home  vs.  away)? 

•  Shaping  choices  of  countries  at  strategic  crossroads 

♦  Which  are  the  key  countries  we  need  to  or  can  shape? 

♦  What  does  the  DoD  have/need  to  accomplish  shaping  and  who  are  our 
partners? 

♦  How  should  we  allocate  resources  across  competing  objectives? 

•  Preventing  the  acquisition  or  use  of  WMD 

♦  What  are  the  investment  priorities?  How  much  is  enough? 

♦  What  are  our  adversaries'  measures  of  success? 


How  does  DoD  balance  capabilities  and  risks 


with  limited  resources? 


I71SH5 
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In  order  to  help  those  that  are  focused  on  developing  and  delivering  tools  designed  to  inform 
decision  makers,  the  subgroup  teams  developed  a  list  of  generic  questions  that  senior  leaders 
might  ask  analysts  to  answer. 

The  non-traditional  nature  of  current  and  future  security  challenges  drives  questions  that 
extend  beyond  the  “how  many,  how  much”  of  current  planning.  Our  future  analytic  tools 
must  be  able  to  answer  questions  and  provide  insight  on  issues  as  complex  as  return  on 
investment,  balancing  resource  allocation  across  organizational  boundaries,  the  operational 
and  strategic  impact  of  capabilities  and  cost  benefit. 

Future  analytic  tools  must  facilitate  discussion  of  risk  —  technological,  operational  and  fiscal 
—  in  order  to  fully  inform  leadership  regarding  the  impact  of  a  particular  decision. 
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WG  5  —  Each  Focus  Area  Faces  Challenging 
Integration  Issues 
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•  Technical 

♦  Capability  measures 

♦  Data 

♦  Models  and  simulation 

♦  Systems  of  systems  integration  (optimization  across  services,  platforms  and 
CONORS) 

•  Resource 

♦  Matching  tools  to  objectives 

♦  Apportionment  across  AORs 

♦  DoD  and  other  department 

•  Organizational 

♦  Reconciling  innovation  with  integration 

♦  Resistance  to  change  (cultural,  organizational) 

♦  Leadership  buy-in  and  commitment 

•  Political 

♦  Interagency,  allied,  coalition,  NGO 


As  staled  earlier,  integration  provides  a  clear  path  to  better  functional  capability  and 
performance  against  the  challenges  presented  in  a  non-traditional  environment.  Working 
Group  5  identified  several  challenges  that  must  be  overcome  in  order  to  develop  effective  and 
efficient  decision  making. 

Technical  issues,  driven  by  data  and  metrics,  and  resource  questions  must  be  answered  to 
provide  the  foundation  for  sound  research,  development  and  procurement  investment 
decisions.  Cultural  and  organizational  biases  present  challenges  to  integration,  and 
leadership  s  support  of  the  overall  objective  will  be  key  to  overcoming  resistance  to  change. 

Developing  closer  relationships  and  opening  effective  lines  of  communication  will  be  an 
important  step  in  attaining  the  integration  required  to  improve  decision  making. 
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WG  5  -  Defeat  Terrorist  Network  Capabilities 

_ _ Wljtwv  op*r*ttorw  toMarrh 

1,  **  Human  intelligence  to  discern  the  intentions  of  the  enemy. 

2,  Persistent  surveillance  to  find  and  precisely  target  enemy  capabilities  in  denied  areas. 

3.  Capabilities  to  locate*  tag,  and  track  terrorists  in  all  domains  Including  cyberspace. 

4.  Special  Operations  Forces  to  conduct  direct  action,  foreign  internal  defense*  counter  terrorist 
operations  and  unconventional  warfare. 

1  Multipurpose  forces  to  train,  equip,  and  advise  indigenous  forces*  deploy  and  engage  with  partner 
nations,  conduct  irregular  warfare,  and  support  security  stability  transition  and  reconstruction 
operations. 

e.  ”  Capabilities  and  organizations  to  help  fuse  intelligence  and  operations  to  speed  action  based  on 
time  sensitive  intelligence, 

7.  "  Language  and  cultural  awareness  to  facilitate  the  expansion  of  partner  capacity, 

a.  Non  lethal  capabilities. 

s.  Urban  warfare  capabilities. 

Prompt  global  strike  to  attack  fleeting  enemy  targets  rapidly, 

it.  Riverine  warfare  capabilities  to  Improve  the  ability  of  US  forces  to  work  with  the  security  forces  of 
partner  countries  to  deny  terrorist  groups  the  use  of  waterways. 

12. **  Ability  to  communicate  US  actions  effectively  to  multiple  audiences,  while  rapidly  countering 
enemy  agitation  and  propaganda, 

13.  **  Joint  coordination,  procedures*  systems*  and  when  necessary,  command  and  control  to  plan 
and  conduct  complex  Interagency  operations. 

14.  **  Broad  flexible  authorities  to  enable  the  United  States  to  rapidly  develop  the  capacity  of  nations 
to  participate  actively  in  disrupting  and  defeating  terrorist  networks. 


Need  ca  pa  b  i  1  i  ty  mea  s  urea  * 


QDR  2005  identified  14  key  capabilities  required  to  defeat  a  terrorist  network.  Of  that  list, 
our  working  group  believed  that  six  capabilities  present  challenges  for  analysts  and  model 
developers  due  to  the  absence  of  appropriate  measures  of  effectiveness.  (. Indicated  with  ** 
before  the  capability.) 
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WG  5  -  Focus  Area:  Defeat  Terrorist  Network  ||1©3S  i 
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Assess  the  QDR  capabilities  -  High  Impact 

♦ 

*’  Language  and  cultural  awareness  to  facilitate  the 
expansion  of  partner  capacity 

Cost  Avail&biHtv 

Low  Near 

♦ 

“  Ability  to  communicate  U  S.  actions  effectively  to  multiple 
audiences,  while  rapidly  countering  enemy  agitation  and 
propaganda 

Low 

Near 

♦ 

**  Human  intelligence  to  discern  the  intentions  of  the  enemy 

Low 

Mid 

♦ 

Prompt  global  strike  to  attack  fleeting  enemy  targets  rapidly 

High 

Near 

♦ 

"  Capabilities  to  locate,  tag,  and  track  terrorists  in  all 
domains  including  cyberspace 

High 

Far 

Need  capab i 1 ity  m eas u re s . 

^ - - - - - - - 

Of  the  six  capabilities  which  we  assessed  as  not  having  well  developed  metrics,  four  are 
assessed  as  having  high  impact  on  force  planning.  (Indicated  with  **  before  the  capability .) 

Analytical  effort  will  be  required  to  develop  measures  for  these  capabilities.  Without  clear 
capability  measures,  our  efforts  to  improve  these  capabilities  may  not  be  effective  or 
efficient. 
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WG  5  -  Focus  Area:  DHiD  mens 
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•  Assess  the  QDR  capabilities  for  DHiD  [Cap=Oapahility] 

Capl  Joint  Cap2  for  HLD  and  Civil  Support 
Cap2  Air  and  Maritime  Domain  Awareness 
Cap3  Consequence  Management 
Cap4  Medical  Countermeasures 
Caps  Tailored  Deterrence 
Cap6  New  /  expanded  authorities 


•  Capabilities  that  will  influence  force  sizing  (in  order  of  potential 
impact) 

♦  Cap2  and  Cap3  may  have  the  same  impact,  but 

*  Cap2  success  depends  on  building  actionable  awareness 

#  Cap 3  acknowledges  inevitability  of  an  event,  and  value  of  Cap 3  in 
other-than-HLD  capabilities 

♦  Cap6  impacts  could  be  significant  if  guard /re  serve  authorities  are  adjusted 

♦  Cap5  could  imply  significant  force  (re  )sizing/re  balancing.  depending  on 
CONORS 

♦  Cap4  could  result  in  rebalancing  organizations,  end-strength  and  skill  sets 

♦  Capl  is  really  a  policy/  ailgnment/resource  issue,  not  technology  or  force 
structure 


Need  capability  measures. 


More 


3 

m 

c 


Less 


QDR  2005  identified  six  key  capabilities  required  to  provide  Defense  of  the  Homeland  in 
Depth  (DHiD), 


The  order  reflects  the  degree  of  influence  we  believe  they  would  likely  have  on  force  sizing 
decisions.  In  this  context,  we  expect  Air  and  Maritime  Domain  Awareness  and  Consequence 
Management  (CM)  to  have  roughly  comparable  implications  for  force  structure. 

Additionally,  investments  in  Consequence  Management  capabilities  would  likely  provide 
further  value  in  non-military  applications  (e,g,,  environmental  incidents). 

Consequence  management  may  be  a  capability  that  drives  force  planning  to  a  high  degree. 
Therefore,  initiatives  to  develop  metrics  to  measure  that  capability  should  be  undertaken  as 
soon  as  possible.  Measures  for  deterrence  capability  may  be  leveraged  from  work  ongoing  at 
STRATCOM, 
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WG  5  —  Focus  Area:  Shaping  Choices  of 
Countries  at  Strategic  Crossroads 

•  AM  QDR  capabilities  will  influence  force  sizing 

♦  Shaping  to  achieve  cooperation 

-*  Security  cooperation  (training,  military  to  military  coop.  etc  .  J  with  allies  and  potential 
partners 

■*  Language  and  cultural  awareness  training  (content,  instructors,  trainees) 

♦  Shaping  to  deter  competition  and/or  attack 

*  Persistent  surveillance 

+  Rapid  mobility  end  power  projection 

+  increased  prompt  global  strike 

■*  Secure  broadband  into  contested  areas  (GIG) 

*  \  nteg  rated  ball  I  stic  a  nd  cruise  missi  I  e  ctef  en  ses 

*  Air  dominance 

+  Undersea  warfare 

Shape  and  defend  cyberspace 

_ -»  Join!  survivable  C2  _ _ 

•  Additional  shaping  capabilities 

♦  Net  assessment  to  support  long  term  planning 

♦  Effective  strategic  communications  (military,  non-military) 

-*  Use  tO  to  influence  risk  perception 

♦  I  n  creased  e  ng  agem  en  t  i  n  host  cou  ntries 


Need  capability  measures. 
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The  WG  5  subgroup  team  assigned  to  examine  capabilities  required  to  effectively  shape 
countries  at  strategic  crossroads  found  that  force  sizing  will  be  influenced  by  the  entire  set  of 
capabilities  presented  in  the  QDR.  Of  these  1 1  key  capabilities,  this  group  found  four  for 
which  they  believe  measures  do  not  exist.  Of  note,  two  of  these  will  require  exploration  of 
metrics  influenced  by  the  social  sciences. 

In  addition,  this  subgroup  listed  several  additional  capabilities  that  could  enhance  the 
effectiveness  of  shaping  efforts,  three  of  which  are  listed  on  this  slide.  To  enable  better 
strategic  and  long-term  planning,  better  net  assessment  capabilities  are  required.  Strategic 
communications  capability  — -  to  include  information  operations  —  may  be  appropriate  to 
influence  our  adversary’s  perception  of  risk  and  allow  us  to  influence  his  decision  making. 
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The  WMD  subgroup  team  used  a  systems  engineering,  process  and  function  driven  approach. 
To  provide  operational  context  and  a  framework  for  discussion,  the  group  presented  a  notional 
supply  chain  that  begins  with  the  gathering  of  knowledge  and  materiel,  and  proceeds  through  a 
series  of  steps  ending  with  CM  requirements. 

To  focus  on  force  sizing  implications,  the  team  described  the  functional  capabilities  that  might 
be  required  to  address  this  challenge,  including  deterrence,  detection,  interdiction  and 
consequence  management.  To  provide  insight  to  DoD  leadership,  the  group  delineated  the 
areas  of  responsibility  (between  civil  authorities  and  the  DoD,  as  currently  defined)  to  focus 
ownership  for  any  capability  investment  that  might  be  required. 

This  construct  provides  the  foundation  for  discussion  of  a  methodology  that  will  permit  issue 
deconstruction  followed  by  cost-benefit  and  risk  analysis.  Within  this  framework,  the  decision 
maker  can  establish  priorities  (i.e.,  which  part  of  the  supply  chain  should  we  focus  on)  and  then 
balance  cost  versus  risk  more  effectively. 
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WG  5  -  Focus  Area:  Preventing  the 
Acquisition  or  Use  of  WMD 


mens 
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After  developing  the  WMD  supply  chain,  the  team  developed  a  value  hierarchy  construct  for 
the  WMD  focus  area.  The  value  hierarchy  portrays  a  potential  decision- framing 
methodology.  However,  this  methodology  could  be  expanded  to  create  an  overall  value 
hierarchy  that  could  include  all  four  focus  areas  in  a  single  value  hierarchy.  To  be  complete, 
the  hierarchy  requires  expansion  using  experts  to  identify  performance  attributes  and 
measures, 

The  Focus’  element  identifies  the  four  QDR  focus  areas  as  an  organizing  construct  for  the 
subsequent  functions  and  performance  characteristics.  These  characteristics  would  require 
definition  such  that  mission  value  or  stakeholder  priorities  are  explicit. 

“Functional  Capabilities”  are  those  that  support  the  performance  or  achievement  of  a 
particular  focus  area.  In  this  example,  there  is  a  need  to  deter,  detect,  interdict  and  manage 
consequences  of  WMD  activities  and  events.  The  next  level  assigns  “Performance 
Attributes”  to  each  functional  capability,  defining  the  necessary  tasks  that  must  be  performed 
in  order  to  arrive  at  a  particular  functional  capability.  Finally,  “Performance  Measures”  are 
developed  to  permit  analysis  of  specific  systems  or  technology  solutions. 

Adopting  an  overall  approach  would  permit  discussion  of  the  relative  value  of  particular 
capabilities  provide  a  basis  for  investment  and  resource  allocation  discussions. 
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WG  5  -  Focus  Area:  Preventing  the 
Acquisition  or  Use  of  WMD 
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•  Assess  the  QDR  capabilities  - 


♦ 


Detection,  interdiction,  and  management  capabilities  may  enhance  deterrence. 


Identify  capabilities  that  will  influence  force  sizing 

iODft  Capabilities 


SOF  locate  /  characterize/  secure 

Locale  /  Tag  /  Track _ _ 

Detect  fissile  malarial  at  standoff 
range _ _ _ 

.Air  /  maritime  /  ground  interdiction 

iRersistenl  surveillance  over  wide 
areas _ 

HUMINT  /  language  skills  /  cultural 
awareness 


Teams  to  render  safe  /  secure 

Non  Lethal  weapons  to  secure 
sites 


JC2 _ _ _ 

Deploy  i  sustain  /  support  /  protect  / 
iredeploy  SOF  hostile  environs 

■  Shield  critical  systems  / 
'technologies  from  EMP 


Finally,  the  WMD  subgroup  considered  which  of  the  eleven  capabilities  highlighted  by  the 
QDR  report  had  implications  for  force  planning. 

This  slide  lists  each  capability,  arrayed  against  functional  capabilities  derived  from 
preliminary  WMD  supply  chain  analysis.  Blue  boxes  (— )  indicate  that  the  functional 
capability  will  influence  force  sizing.  For  example,  investment  in  HUMINT,  language  skills 
and  cultural  awareness  would  improve  our  ability  to  detect  WMD  as  well  as  manage 
consequences.  Adding  SOF  capability  would  improve  detection  and  interdiction  ability.  In 
any  case,  increased  capabi  lity  would  provide  the  ancillary  benefit  of  increasing  risk  to  the 
adversary,  enhancing  deterrent  capability. 
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WG  5  -  Conclusions  lil6iR5 
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•  Future  makes  scenario  based  planning  more  important 

♦  Uncertain  and  dangerous 

♦  Complex,  intelligent,  adaptive  threats 

•  Larger,  more  diverse  group  of  decision  makers  and 
stakeholders 

♦  Difficult  to  trade-off  between  diverse  capabilities 

♦  Fundamental  issue  may  be  balancing  risk 

•  Non-traditional  complexity  present  new  integration  challenges 

♦  Many  complexities  caused  by  threats,  distributed  decision  making,  and 
very  large  number  of  stakeholders 

♦  Difficult  to  integrate  priorities  across  non-traditional  security  areas 

•  Defining  functions  of  each  focus  areas  will  help  measure,  model 
and  simulate  QDR  capabilities 

♦  We  need  measures  for  some  of  the  most  important  capabilities 

Makes  Title  10  provider’s  job  more  challenging. 


In  conclusion,  Working  Group  5  agreed  that  the  uncertain  and  potentially  dangerous  future 
requires  the  use  of  scenario-based  planning.  The  scenarios  must  be  creative  enough  to  enable 
us  to  think  like  our  adversaries.  The  ability  to  analyze  complex  and  adaptive  adversaries  will 
be  critical,  and  analysis  augmented  by  “red  cell  insights  and  the  social  sciences  will  be 
necessary. 

Additionally,  the  group  highlighted  the  increasing  complexity  of  decision  making. 
Competition  for  scarce  resources  driven  by  individual  and  organizational  perspectives  and 
priorities  will  make  trades  more  challenging. 

Integration  is  critical,  particularly  in  the  face  of  the  broad  challenges  presented  by  an 
intelligent,  committed  adversary  employing  unconventional  methods. 

Finally,  defining  functions  will  help  develop  improved  measures  and  models.  In  order  to 
provide  appropriate  resources  to  important  capabilities,  we  need  to  develop  measures  and 
methods  to  adequately  describe  potential  return  on  investment. 
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Risk  Analysis  May  Be  a  Useful 
Framework  for  Defense  Analysis. 
Illustrative  Operational  Risk. 
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■ 


Pre-Event 


Ml 


Event 


Post-Event 


Prepare 


Respond 


Restore 


is 


•.'Risk' Assessment  | 
Risk.  Management.^  . 


Use  Existing  Consequence  M 


impact  systems  in  design. 


Since  one  of  our  findings  was  that  risk  management  will  be  critical  in  two  of  the  four  focus 
efforts,  we  used  this  slide  to  introduce  the  risk  management  lexicon. 

Existing  security  systems  (fences,  firewalls,  etc.)  prevent,  identify  and  reduce  known 
vulnerabilities.  These  systems  may  help  reduce  the  probability  of  success  given  an  attack. 

Existing  consequence  management  systems  reduce  the  consequences  of  an  attack  given  a 
successful  attack. 

Risk  analysis  =  risk  assessment  +  risk  management.  The  purpose  of  risk  assessment  is  to 
assess  risk  before  an  event  so  we  can  make  risk  management  decisions  to  improve  our 
security  and  consequence  management  systems. 

The  standard  definitions  for  these  terms  can  be  found  at  the  Society  for  Risk  Analysis  (SRA). 
(2005).  “Glossary  of  Risk  Analysis  Terms.”  http://sra.on’/resources  uiossarv.php 
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WG  5  -  Recommendations  and  Next 
Steps 
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•  Potential  Resource  Allocation  Methods* 

♦  New  focus  on  risk  analysis  methods 
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•  Next  steps 

*  For  each  QDR  focus  area  and  across  the  focus  areas 
■*  Use  creative  new  scenarios 
Define  functions 

-*  Identify  capabilities  to  perform  functions 

*  Develop  measures  for  each  capability 

-*  Develop  models  and  simulations  to  calculate  measures 


*  Caveat  -  WG  S  did  not  examine?  snnr:tflr  mnHfeic  inH  cimuhiin 


A  survey  of  the  27  experts  in  Working  Group  5  produced  the  results  presented  in  the  table 
above.  The  challenges  of  the  non-traditional  environment  will  drive  us  to  examine 
combinations  of  traditional  and  newly-developed  analytic  tools  and  decision  aids.  While 
some  of  the  focus  areas  can  use  current  operations  research  techniques,  others  may  require  a 
different  perspective  —  some  of  which  may  necessitate  education  of  the  decision  maker.  For 
example,  capabilities  required  to  successfully  defend  the  homeland  or  prevent  use  of  WMD 
may  be  better  examined  by  using  risk  analysis  and  value  hierarchies  as  illustrated  in  previous 
slides. 

Working  Group  5  recommends  a  five  step  approach  to  improving  analytic  support,  and  by 
extension,  enhancing  the  ability  of  the  decision  maker  to  make  informed  choices.  This 
approach  would  involve  use  of  creative  scenarios,  well-defined  functions,  capabilities  that 
support  the  functions  and  measures  to  evaluate  performance.  Once  this  precursor  work  is 
accomplished,  the  development  of  models  and  simulations  should  be  pursued. 
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Analysis  for  Non-Traditional 
Security  Challenges: 
Methods  &  Tools 


Military  Operations  Research  Society  (MORS) 
Workshop 


Synthesis  Group 
Outbrief 


This  is  the  Synthesis  Group  outbrief  for  the  Analysis  for  Non-Traditional  Security 
Challenges:  Methods  and  Tools  Workshop  held  at  John  Hopkins  University 
Applied  Physics  Laboratory  in  Maryland  on  21-23  February  2006. 
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Agenda 
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•  Workshop  goals,  panel  goals  and  composition 

•  Insights  on  the  nature  of  the  problem 

•  Selected  findings,  recommendations 

•  Summary 


The  Synthesis  Group  was  formed  to  look  across  the  other  working  groups.  Our 
final  report  brief  will  follow  a  different  outline. 


Workshop  Objectives  I7I6R5 

0«rt(5«w  fttturct,  icom 

•  Illuminate  users  of  non-traditional  modeling  and 
simulation  gaps  and  deficiencies  in  the  context  of  the 
defense  security  challenges  and  emerging  changes  in 
the  defense  strategy 

•  Identify  areas  of  user  commonality 

•  Capitalize  on  Joint,  Service  and  Agency  toolsets  and 
ongoing  model  development  for  the  larger  community 

•  Develop  partnerships  to  share  future  M&S 


These  are  the  goals  for  the  workshop  from  the  Terms  of  Reference  and  as  briefed  to  the 
MORS  Sponsors  involved  in  workshop  planning. 


Workshop  Guidance 
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•  Get  the  right  match  of  stakeholders,  technologists 
and  thinkers  into  each  WG 

•  Encourage  participation  by  innovative  people  with 
new  ideas  and  new  tools 

•  Shape  the  meeting  towards  the  goal  of  investing  in 
commonly  usable  products  to  advance  our  ability  to 
aid  decision  making 


These  points  were  the  additional  guidance  and  points  of  emphasis  the  Program 
Committee  received  from  the  MORS  Sponsors, 
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Panel  Goals,  Objectives,  Scope 
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•  Goals 

♦  Provide  an  overview  of  the  entire  workshop 

•  Objectives 

♦  Clarify  the  nature  of  the  problem  by  conducting  internal  panel 
discussions 

♦  Capture  the  framing  issues  based  on  the  presentations  at  the 
Plenary 

♦  Derive  key  themes,  findings,  and  recommendations  from  the 
results  of  the  individual  panels 

•  Scope 

♦  Focus  on  support  for  Strategic/Departmental  level  decision 
making 

♦  Future  challenges  and  recommendations 


By  definition,  the  working  groups  are  not  mutually  exclusive.  The  inherent 
overlaps  between  the  working  groups  provides  synthesis  points  for  integrating 
the  conclusions  from  each;  as  well  as  reducing  the  probability  that  major  ideas 
will  fall  through  the  cracks  between  the  workshop  topics.  The  Synthesis  Group 
examined  cross-group  commonalities  and  insights  and  developed  additional 
workshop  recommendations. 

Consistent  with  the  goal  to  provide  an  overview  of  the  entire  workshop,  the 
Synthesis  Group  pursued  three  supporting  objectives.  First,  it  sought  to  clarify 
the  nature  of  the  problem.  Second,  as  a  group,  it  reviewed  the  presentations  from 
the  plenary  session  for  cross  cutting  issues  and  themes  and  used  those  results  to 
aid  in  it’s  review  of  WG  findings.  Finally,  the  group  derived  key  findings  and 
recommendations  based  largely  on  an  integration  across  the  results  of  the 
individual  working  groups. 


109 


Recruiting  Was  Tough... 
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As  many  of  you  know,  members  of  the  Synthesis  Group  have  a  day-time  job  and 
a  night-time  job.  During  the  day,  each  member  of  the  group  was  embedded  in 
one  of  the  four  mission  oriented  working  groups.  During  off-hours  we  met  to 
share  insights  and  develop  a  holistic  view  of  the  subject.  Thus,  recruiting  can  be 
difficult,  requiring  unorthodox  approaches. 
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Synthesis  Panel  Composition 


Itishs 


•  Renee  Carlucci  -  MCCDC 

•  Sunny  Conwell  -  N81  (Synthesis  Panel  Co-chair) 

•  Lee  Dick,  FS  -  N81  (Workshop  Chair) 

•  Jeff  Hamman  -  JHU  APL 

•  Jim  Harris  -  AF/A9  (Workshop  Co-Chair) 

•  Greg  Keethler  -  Lockheed  Martin 

•  Roy  Reiss  -  AF/A9  (Synthesis  Pane)  Co-chair) 

•  Roy  Rice,  FS  -  Teledyne  Brown  Engineering 

•  Vince  Roske,  FS  -  Institute  for  Defense  Analyses 

•  Bill  Sentlinger  -  OSD/JDS 


The  Synthesis  Group  was  assigned  to  working  groups  as  shown  with  some 
members  assigned  to  float  between  working  groups.  Multiple  people  were 
assigned  to  WG  1  and  WG  5  since  these  working  groups  subdivided  into  multiple 
teams  to  focus  on  specific  topics. 

•  Renee  Carlucci  -  WG  3 

•  Sunny  Conwell  -  WG  1 

•  Lee  Dick,  FS  -  float 

•  Jeff  Hamman- WG  5 

•  Jim  Harris  -  float 

•  Greg  Keethler  -  WG  5 

•  Roy  Reiss  -  float 

•  Roy  Rice,  FS  -  WG  4 

•  Vince  Roske,  FS  -  WG  1 

•  Bill  Sentlinger- WG  1 
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Speaker  Perspectives 
From  the  Plenary 
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•  Keynote  -  VADM  Chanfk 

♦  We  are  re-balancing  future  force  capabilities 

♦  "Fighting  the  Long  War”  focuses  on  key  points  that  have  not 
been  addressed  in  traditional  warfare  analysis  —  building 
relationships  and  early  preventive  measures.  How  these  are 
addressed  in  M&S  tools/methods  is  a  critical  question 

♦  In  the  current  security  environment  success  requires  integration 
of  all  US  Government  capabilities  and  greater  cooperation  with 
allies  and  partners 

♦  The  QDR  has  specific  end  states  desired 

How  are  these  evaluated  using  traditional  tools  and  methods? 

How  differently  are  these  non-traditional  warfare  questions  to 
previous  traditional  warfare  questions? 


During  the  plenary  session  initiating  the  event,  eight  distinguished  speakers 
addressed  several  key  issues. 

The  following  section  briefly  summarizes  perspectives  on  those  issues.  It  is 
meant  to  capture  selected,  major  points  articulated  by  the  speakers  rather  than 
provide  a  comprehensive  summary  of  their  remarks. 

VADM  Chanik’s  keynote  "Meeting  the  Nation 's  Challenges  of  Tomorrow 
How  the  Operations  Research  Analyst  Can  Help,  ”  provided  insights  to  shape 
working  group  discussions. 

He  also  identified  five  challenges  for  the  participants  in  the  workshop  and  for  the 
Defense  analytic  community: 

1.  Continue  to  broaden  open  and  collaborative  environments  to  study  issues 
concerning  the  Department  of  Defense. 

2.  Continue  the  work  started  in  past  workshops  and  the  analytical 
methodology  working  group. 

3.  Identify  required  capabilities  to  fight  our  wars  in  the  future. 

4.  Develop  analytical  methodologies  to  analyze  non-traditional  warfare. 

5.  Provide  a  list  of  gaps  in  tools  and  methods  for  development  within  the 
research  community. 
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Perspectives  from  the  Plenary  fil635 
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*  Applicability  of  Operations  Analysis  Techniques  -  Dr  Haniey 

♦  Traditional  OA  techniques  do  apply  to  the  n on-traditional  challenges 

+  Multidfsplinary  teams 

-*>  Analytic  Methods:  Operational  Statistics,  Operational  Experiments,  Analytic 
Wargaming;  Complex  Sciences,  Military  Soft  Power  (Social  and  Economic 
Sciences) 

"Hemibel  ttiinking*  —  Factor  of  3  improvement 

♦  Criteria  for  investing  in  simulations  of  soft  military  power 

*  internal  Validity 
■*  Usability 
■*  Extend ibility 
**  Replication  of  results 

*  Cyber  Warfare  -  Dr  Kass 

♦  Cyberspace  is  the  only  facet/environment  in  which  the  US  has  peer  competitors 

♦  It  is  the  only  entirely  manmade  environment  that  we  operate  In 

*  2006  Quadrennial  Defense  Review  ^Col  Kelly 

♦  integrated,  augmenting,  and  complementary  analysis 

♦  Employ  a  broad  array  of  analytic  methods  and  tools 
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Perspectives  from  the  Plenary 

•  Service  Perspectives 

♦  Air  Force  -  Dr  Jacqueline  Henningsen,  FS 

The  A ion~Trvdftionat  Security  Challenge  requires  us  to  think  of  our  enemies  and 
ourselves  as  a  part  of  a  complex  system  —  With  many  interdependent  parts 

**  To  analyze  a  complex  system,  we  must  first  understand  its  behavior  —  Not  Only 
behavior  of  the  parts.,  .but  how  they  form  the  behavior  of  the  whole 

♦  Navy  -  Mr.  A-H.  Barber  III 

+  What  methods,  techniques  and  tools  can  be  applied  to  answer  questions 
associated  with  non -traditional  challenges? 

->  How  do  we  get  trusted  results  that  decision  makers  are  willing  to  use? 

*  Where  we  do  not  have  accepted  methods  and  tools,  how  do  we  fill  the  gaps? 

♦  USMC  -  Dr.  George  Akst 

*  The  inability  to  assess  the  interdependency  of  military,  civilian,  and  multi¬ 
national  efforts  during  the  conduct  of  stability  operations  Is  the  major  analytic 
shortcoming 

-¥  Success  needs  to  be  clearly  defined  and  translated  into  quantitative  MOEs 
before  analysts  can  build  models 
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Perspectives  from  the  Plenary  li)€>H5 

*  MMtarv  Dptfilioii  Fi  u  ma nlii  Society 

•  Service  Perspectives 

♦  Army  -  Dr  Markowitz 

■*  Strategy  Mix  and  Integration 

-  Scenario  Mix;  Force  Management;  Force  Planning;  Acquisition; 
Stationing  and  Integrated  Global  Posture  and  Basing 

-  Each  area  requires  different  analysis  techniques 

■*  Common  themes:  complex  environment,  non  combatants,  C4ISR 

•  Common  Themes  and  Additional  Insights 

♦  This  is  not  a  new  problem  but  it  is  still  not  well  understood 

♦  Need  to  understand  the  problem  before  building  the  model 

♦  Success  requires  integration  of  all  USG  capabilities  and  greater 
cooperation  with  allies  and  partners 


One  major  theme  from  multiple  speakers  was  that  this  is  not  a  new  problem  the 
DoD  faces.  It  is  a  problem  that  the  defense  analytic  community  and  MORS  has 
addressed  before,  but  the  recent  past  has  not  been  a  major  analytic  focus  area.  It 
is  also  an  area  that  is  not  well  understood  and  more  thought  on  the  nature  of  the 
problem,  understanding  of  the  decision  space,  measures  of  success  and  metrics  is 
needed  before  detailed,  trusted,  models  can  be  built. 

Finally,  several  of  the  speakers  pointed  out  that  successful  analysis  needs  to  take 
into  account  the  capabilities  of  the  other  non -defense  organizations  that  play  a 
key  role  in  these  non-traditional  challenges  and  scenarios  and  how  their  activities 
integrate  with  defense  activities. 
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The  Working  Groups: 

An  Assessment 

♦  Did  we  get  we  get  the  right  match  of  stakeholders, 
technologists  and  thinkers  into  each  WG? 

4  Overall  the  workshop  had  a  diverse  group 

With  diversity  comes  more  time  reaching  a  common  frame  of  reference 

*  Involving  the  other  government  organizations  a  work  in  progress 

♦  Working  Group  pre-workshop  preparations 

Catastrophic  WG:  Stakeholder  surveys  to  frame  discussion 
Deterrence  WG:  Low  response  rate  a  problem 

*  Irregular  WG:  Innovative  use  of  scenarios  to  focus  discussion 
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One  of  the  key  points  of  emphasis  for  the  workshop  sponsors  was  to  get  the  right 
match  of  stakeholders,  technologists  and  thinkers  into  each  WG  and  to  encourage 
participation  by  innovative  people  with  new  ideas  and  new  tools. 

The  workshop  did  achieve  a  diverse  group  of  participants,  as  shown  on  the  next 
slide.  There  were  approximately  1  S3  attendees  with  83  of  them  being  new  to 
MORS.  The  organizing  committee  worked  hard  to  bring  in  folks  from  other  non- 
defense  agencies,  allies  and  the  commercial  sector  who  do  not  normally  work  on 
traditional  defense  analysis  issues. 

However  achieving  a  diverse  group  of  participants  meant  that  the  working  groups 
needed  to  spend  more  time  in  the  first  part  of  their  sessions  making  sure  that  all 
of  the  members  had  a  common  set  of  definitions  and  frame  of  reference.  In 
addition,  despite  the  committee  efforts,  all  of  the  working  groups  wanted  more 
participation  from  the  non-defense  government  organizations. 

Three  of  the  working  groups  tried  to  do  innovative  pre- workshop  preparation. 
Two  tried  pre-workshop  surveys  to  help  focus  their  efforts.  WG  3  (Catastrophic) 
found  it  useful  while  WG  4  (Deterrence)  did  not  due  to  a  low  response  rate.  WG 
1  (Irregular/GWOT)  developed  a  group  of  scenarios  that  were  used  at  the 
workshop.  Teams  were  assigned  to  analyze  each  scenario  to  see  what  analytic 
method  and  tools  they  might  use  if  assi  gned  a  study  in  that  area. 
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Attendee  Demographics 
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The  Working  Groups: 
Common  themes 
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*  Scoping  DoD  mission 

♦  In  some  cases  DoD  is  supporting  organization 

♦  Defining  capabilities  expected  from  DoD 

*  What  are  decision  maker's  key  issues  and  what  are  the  suitable  measures 

*  Defining  interaction  with  other  government,  NGOs  and  international 
organizations 

♦  Roles  and  responsibilities 

♦  Different  organizational  structures  and  cultures 

♦  information  and  data  providers 

*  Methods 

♦  Analytic  Wargaming 

♦  Non-M&S  methods  (risk  analysis,  cost-benefit  analyst,  social  networks, 
game  theory) 

♦  Integration  across  an  ensemble  of  models/tools 


Scoping  the  DoD  mission;  The  DoD  role  in  many  of  the  non-traditional  areas  is  in  support  of 
non-defense  organizations.  The  capabilities  required  to  support  or  augment  other  organizations 
will  need  to  be  well  defined. 

Key  issues;  Analysts  need  to  focus  on  the  key  issues  and  defining  the  problems  and  impacts  of 
decisions  on  resource  allocation,  rather  than  rush  to  develop  models,  Non-traditional  challenges 
may  take  more  time  to  develop  up  front. 

Interactions  with  non-DoD  organizations:  The  DoD  has  collaborated  to  develop  the  UJTL 
(Uni  versal  Joint  Task  List)  to  provide  a  common  lexicon  for  defining  the  capabilities  of  the 
Services,  There  are  US  government  efforts,  such  as  USAID,  to  collaborate  with  other 
governments,  I G Os.  and  NGOs  to  develop  international  definitions.  The  DoD  will  need  to  be 
active  in  this  lexicon  development. 

The  DoD  organizational  structure  with  respect  to  regions  of  the  world  is  very  different  from  other 
organizations.  For  example,  DoD  has  3  COCOMs  that  deal  with  Africa,  where  most  organizations 
have  a  unique  interaction  with  Africa. 

Data  from  non  DoD  organizations  will  be  necessary  for  NTW  analysis,  MG  As  and  MOlJs  will  be 
needed  to  establish  trusted  and  protected  data  sources  for  this  analysis. 

Methods:  Wargaming  is  a  tool  that  is  well-suited  for  examining  issues  in  NTW.  There  are 
analytic  methods  in  other  communities  that  can  be  applied  to  this  problem  space  such  as  risk 
analysis,  systems  engineering,  or  economic  models* 

DoD  analysts  have  long  used  a  range  of  models  based  on  physics.  However  much  of  the  NTW 
analysis  will  be  based  on  complex  interactions  between  humans.  Even  though  many  models  and 
techniques  were  discussed,  there  wras  a  consensus  that  the  NTW  solutions  will  be  more 
comparative  rather  than  absolute  and  will  require  reasoning  across  an  ensemble  of  models,  w  here 
the  models  are  in  different  domains  (like  the  components  of  PMESII). 
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The  Working  Groups: 
Common  themes 
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•  PEOPLE 


♦  Need  more  diverse  analytic  teams  with  more  social  science  experience 

•  DATA 

♦  Where  is  it?  How  to  acquire?  What  is  available? 

♦  Interaction  between  data,  models  and  metrics 

•  V&V 

♦  How  to  accomplish  V&V  for  new  tools 

♦  Need  to  develop  ways  to  accomplish  V&V  on  methods  and  data 


People:  A  common  denominator  across  the  working  groups  was  the  recognition  that  the 
military  operations  research  community  (which  is  currently  largely  comprised  of 
mathematicians,  physicists,  ORs,  engineers  and  scientists)  will  need  to  be  augmented  by 
other  disciplines  such  as  sociologists,  anthropologists,  regional  experts,  economists, 
political  scientists,  and  psychologists.  It  will  not  be  sufficient  to  leave  these  disciplines  to 
academia  and  reach  back.  We  need  multi-disciplinary  teams  within  the  DoD  to  work 
collaboratively  on  the  analytical  issues  for  non -traditional  warfare. 

Data:  The  physics-based  data  is  well  understood  by  the  MORS  community.  The  soft 
sciences  are  a  new  domain.  Data  mining  technologies  will  need  to  be  incorporated.  The 
design  of  experiments  will  be  very  important,  Methods  for  collecting  data  will  have  to  be 
validated.  Many  data  sets  will  be  noisy  and  different  techniques  for  analysis  will  be 
required.  Sources  that  are  outside  of  the  norm  will  be  required  such  as  mining 
newspapers,  industry  reports,  global/regional  economic  data,  web  postings,  political 
satire,  academic  publications,  etc.  The  interactions  between  data,  models  and  methods 
will  likely  be  more  challenging, 

V&V;  Verification  and  validation  (V&V)  was  a  common  theme  in  working  group 
discussions.  One  area  was  the  need  for  reasonable  and  sufficient  V&V  of  models  and  the 
difficulty  of  accomplishing  V&V  when  there  is  little  real  world,  historic  or  test  results  to 
form  a  basis  of  comparison.  A  second  area  of  discussion  was  the  need  to  accomplish 
V&V  of  the  analytic  methods  and  the  supporting  data  to  build  the  confidence  of  decisions 
makers  when  presented  with  analysis  using  new  methods  and  tools  and  relying  on  data 
from  that  for  unfamiliar  data  sources. 

When  evaluating  a  model,  at  least  two  broad  standards  are  relevant.  One  is  whether 
the  model  is  consistent  with  the  data.  The  other  is  whether  the  model  is  consistent 
with  the  real  world.  -  Kenneth  A.  Bollen,  Structural  Equations  with  Latent 
Variables 
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The  Problem  is  Not  New 

*  But  it  does  reflect  a  shift 
in  the  PoD*s  priorities 
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•  What  is  non-traditional  in  non-traditional  challenges? 


♦  Interdependence  and  relationships  with  other  non-defense  organizations 

♦  A  complex  system  with  many  interdependent  parts,  influenced  by  many  factors  — 
ours  and  theirs 

♦  Importance  of  perception 

♦  Increased  use  of  social  science  methods 

♦  Less  data  available  (some  capabilities  can  be  difficult  to  measure) 

♦  Link  between  methodology,  metrics  and  decision  makers 


While  issues  associated  with  the  non-traditional  challenges  are  not  new  issues 
they  are  issues  that  have  not  been  a  high  priority  for  the  DoD  for  a  while,  until 
their  recent  reemergence.  As  the  keynote  and  other  plenary  speakers  pointed  out 
it  does  reflect  a  shift  in  the  DoD's  priorities. 

The  second  part  addresses  a  question  raised  by  a  member  of  the  Synthesis  Group 
in  our  initial  discussions  and  by  two  of  the  working  groups  in  their  discussions. 
They  represent  areas  that  are  either  not  critical  to  successful  analysis  of  the 
traditional  Major  Contingency  Operations  (MCOs)  or  are  unique  to  the  non- 
traditional  challenges.  The  relationship  with  non-defense  organizations  is  critical 
to  analysis  on  the  non-traditional  scenarios,  in  some  cases  the  DoD  is  the 
supporting  organization  and  the  needed  capabilities  and  definition  of  success  is 
defined  by  the  supported  organization.  Many  of  the  new  issues  require  a  complex 
systems  approach  where  it  can  be  difficult  to  measure  success.  Much  can  depend 
on  perception  or  the  ability  to  change  or  influence  perception.  In  some  cases 
success  is  no  action  or  activity.  This  includes  greater  use  of  social  science 
methods  and  data  associated  with  those  methods.  New  data  sources  need  to  be 
identified  and  in  many  cases  the  data  will  be  more  noisy  or  sparse  than  more 
hard  data  such  as  weapon  system  dates  that  are  used  for  analysis  of  MCOs. 

Finally,  as  we  introduce  new  methods  and  tools  it  will  take  time  for  the  decision 
makers  to  develop  the  level  of  understanding  and  comfort  they  have  in  studies 
using  the  methods,  tools  and  models  they  are  familiar  with  from  years  of  seeing 
the  results  of  studies  using  the  MCO  based  scenarios. 
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In  Summary 


m©as 

Ornitioni  ftw wth  Sonet) 


*  Accomplishments 

♦  The  workshop  made  progress  against  objectives 

4  Review  of  current  tools 

**  Identify  gaps  In  skill  sets  and  requirement  for  more  diverse  OR  teams 

♦  i  o  paraphrase  the  psychiatrist  in  Portnoy's  Complaint:  “Now,  we  are  ready  to 
begin.*," 

•  Residual  Challenges 

♦  Reach  out  to  a  broader  community  (e*g.,  Inter-agency  and  coalition)  continue  to 
broaden  open  and  collaborative  environments 

♦  Regard  this  workshop  as  part  of  a  sequence,  continue  the  work  started  in  the 
past  workshops  and  analytical  methodology  working  group 

♦  Continue  to  develop  analytical  methodologies  to  analyze  n  on-traditional 
challenges 

♦  Methodologies  to  facilitate  assessing  and  balancing  risk 

♦  W&A  should  address  methodologies  and  data  as  well  as  tools 


The  Synthesis  Group  sought  to  put  the  workshop  into  perspective  by  generating  a  scorecard  to 
summarize  the  positive  accomplishments  of  the  event  and  the  residual  challenges  that  confront  the 
community* 

Accomplishments.  All  of  the  working  groups  recognized  the  fact  that  we  need  to  first  understand  the 
types  of  problems  we  are  attempting  to  solve  and  the  need  to  step  through  a  1  st  order  solution  process. 
Models  may  not  be  required  to  solve  the  problem;  but  if  a  model  is  required,  an  inventory  of  the 
available  models  is  needed.  We  also  have  a  need  to  class  problems  by  common  characteristics  so  that 
we  can  identify  new  and  different  subject  matter  experts  and  technologists  to  assist  in  the  collaborative 
solution  process. 

All  working  groups  realized  that  our  brainpower  resources  must  be  expanded  in  to  areas  where  the 
DoD  has  typically  not  used  frequently.  These  areas  are  in  the  human  behavior,  social,  behavioral,  and 
economic  disciplines.  All  of  the  working  groups  pointed  out  the  compelling  need  to  attract  more 
representatives  from  the  executive  departments,  state  and  local  government  agencies. 

However,  it  must  be  understood  that  this  constitutes  a  beginning,  not  an  end.  Our  status  is  reminiscent 
of  Philip  Roth's  Portnoy  at  the  conclusion  of  his  book  Portnoy 's  Complaint.  After  a  difficult,  soul- 
wrenching  confessional,  his  psychiatrist  concludes  the  book  with  the  words  “Now,  we  are  ready  to 
begin...” 

Residual  Challenges.  Since  we  are  now  ready  to  begin,  we  believe  that  there  are  three  major 
initiatives  that  the  community  should  undertake.  First,  there  is  a  need  to  reach  out  to  a  broader 
community  to  address  the  non -traditional  challenges.  This  includes  enlisting  the  aid  of  the  inter-agency 
and  alliance/coalition  communities.  Second,  we  need  to  realize  this  workshop  is  just  one  in  a  sequence 
of  workshops  that  will  help  the  defense  analysis  community  develop  new  methodologies,  methods,  and 
tools  to  address  the  non -traditional  challenges.  This  includes  developing  methodologies  that  facilitate 
assessing  and  balancing  risk  and  the  appropriate  V&V  for  those  methodologies  that  will  enable 
decision  makers  to  quickly  develop  an  understanding  of  the  resulting  analysis  and  confidence  in  those 
results. 
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Closing  thoughts:  li1©R5 

Quotes  from  the  workshop... 

•  Agent  Based  Modeling: 

♦  “DoO  is  in  the  caboose.  Most  trains  don't  have  a  caboose." 

•  v&v 

♦  “Some  of  this  may  be  like  measuring  ‘elephant  repellent,’  you 
think  it’s  working  because  you  don't  see  any  elephants.” 

•  Data  Polling 

♦  “Even  if  you  had  the  data,  you  wouldn't  have  confidence:  Did 
they  tell  you  what  they  thought,  or  what  they  thought  you 
wanted  them  to  think,  or  what  they  thought  their  government 
wanted  them  to  think?” 
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MORS  Workshop:  Analysis  for  Non-Traditional  Security 
Challenges:  Methods  and  Tools 

JHU/APL,  Laurel,  Maryland 
21  -  23  February  2006 

ACRONYMS 


AFRL 

AMP 

BAE 

C2 

C4ISR 

CAA 

CAPS 

CATS 

CBRN 

CBRNE 

CDC 

CENTCOM 

CIA 

CM 

COA 

COCOM 

COI 

CPB 

CPM 

DARPA 

DART 

DCC 

DCTS/TWS 

DHiD 

DHS 

DIAMOND 

DIME 

DoD 

DOE 

DTO 

DTRA 

EBW 

EPA 

EPICS 

EPS 

ERIS 

ESS 


Air  Force  Research  Laboratory 
Analysis  of  Mobility  Platform 
British  Aerospace 
Command  and  Control 

Command,  Control,  Communications,  Computers,  Intelligence, 

Surveillance  and  Reconnaissance 

Center  for  Army  Analysis 

Commanders  Analysis  and  Planning  Simulation 

Consequence  Assessment  Toolset 

Chemical,  Biological,  Radiological  and  Nuclear 

Chemical,  Biological,  Radiological/Nuclear,  and  Explosive 

Center  for  Disease  Control 

United  States  Central  Command 

Central  Intelligence  Agency 

Consequence  Management 

Course  of  Action 

Combatant  Commander 

Community  of  Interest 

Cultural  Preparation  of  the  Battlefield 

Critical  Path  Method 

Defense  Advanced  Research  Proj  ects  Agency 
Defense  Adaptive  Red  Team  (USSTRATCOM) 

DTA  Collaboration  Center 
Defense  Collaboration  Tool  Suite/  Info  Workspace 
Defense  of  the  Homeland  in  Depth 
Department  of  Homeland  Security 

Diplomatic  and  Military  Operations  in  a  Non- Warfighting  Domain 
Diplomatic,  Information,  Military  and  Economic 
US  Department  of  Defense 
Department  of  Energy 
WG  3  -  Organization?? 

Defense  Threat  Reduction  Agency 

Entropy  Based  Warfare 

Environmental  Protection  Agency 

Emergency  Preparedness  Incident  Command  Simulation 

Engineering  Problem  Statement 

Exploratory  Regional  Insurgency  Simulation 

Engagement  Simulation  System 
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EUCOM 

FDOS 

FEMA 

FFRDC 

GCAM 

GIG 

GWOT 

HITL 

HLD 

HPAC 

HSC 

I/O 

IBC 

IDA 

IGO 

IMEA 

IS 

ITEM 

IWMDT 

J8 

JCATS 

JDS 

JEM 

JFCOM 

JICM 

JMEM 

JOEF 

JRD3 

JSRTS 

JTLS 

JWARN 

JWARS 

M&S 

MCO 

MIT 

MMOG 

MOA 

MOE 

MOU 

MOUT 

MSFD 

NBC 

NGO 

NM 

NORTHCOM 

NPS 


United  States  European  Command 

Flexible  Deterrent  Options 

Federal  Emergency  Management  Agency 

Federally  Funded  Research  and  Development  Center 

General  Campaign  Analysis  Model 

Global  Information  Grid 

Global  War  on  Terrorism 

Human -In- The- Loop 

Homeland  Defense 

Hazard  Prediction  Assessment  Capability 
Homeland  Security  Council 

Input/Output  (example  XML  -  extensible  Markup  Language) 

Integrated  Battle  Command 

Institute  for  Defense  Analyses 

Inter-Governmental  Organization 

Integrated  Munitions  Effect  Assessment 

Israel 

Integrated  Theater  Engagement  Model 

Integrated  Weapons  of  Mass  Destruction  Toolset 

Joint  Staff,  Director  for  Force  Structure,  Resource,  and  Assessment 

Joint  Conflict  and  Tactical  Simulation 

Joint  Decision  Support 

Joint  Effects  Model 

Joint  Forces  Command 

Joint  Integrated  Contingency  Model 

Joint  Munitions  Effects  Matrix 

Joint  Operational  Effects  Federation 

Joint  Rapid  Distributed  Database  Development 

Joint  State  Response  Training  Simulation 

Joint  Theater  Level  Simulation 

The  Joint  Warning  and  Reporting  Network 

Joint  Warfare  Analysis  and  Requirements  System 

Modeling  and  Simulation 

Major  Contingency  Operation 

Massachusetts  Institute  of  Technology 

Massively  Multiplayer  Online  Game  (WG  1 ) 

Memorandum  of  Agreement 
Measures  of  Effecti  veness 
Memorandum  of  Understanding 
Military  Operations  in  Urban  Terrain 
Multi-Service  Force  Deployment 
Nuclear,  Biological  and  Chemical 
Non-Governmental  Organizations 
National  Institutes  of  Health 
United  States  Northern  Command 
National  Planning  Scenarios 
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NSC 

NTW 

OPS 

OR 

OSD 

OSD  (P&R) 

OSD  (PA&H) 

PACOM 

PERT 

PMESII 

PSOM 

QDR 

SD  JOC 

SDM 

SEAS 

SEDP 

SIAM 

SOCOM 

SPG 

STRATCOM 

TIC/TIM 

TOPOFF 

TSC 

UJTL 

UK 

US 

USAID 

USG 

USN 

V&V 

VAPO 

WG 

WMD 


National  Security  Council 

Non-Traditional  Warfare 

Operations 

Operations  Research 

Office  of  the  Secretary  of  Defense 

Office  of  the  Secretary  of  Defense  (Personnel  and  Readiness) 

Office  of  the  Secretary  of  Defense  (Program  Analysis  and  Evaluation) 
United  States  Pacific  Command 
Program  Evaluation  Review  Technique 

Political,  Military,  Economic,  Social,  Infrastructure  and  Information 

Peace  Support  Operations  Model 

Quadrennial  Defense  Review 

The  Strategic  Deterrence  Joint  Operating  Concept 

System  Definition  Matrix 

Systems  Engineering  and  Analysis  Support 

Systems  Engineering  Design  Process 

Situational  Influence  Assessment  Model 

United  States  Special  Operations  Command 

Strategic  Planning  Guidance 

US  Strategic  Command 

Toxic  Industrial  Chemical/Material 

Top  Officials  Three  Exercise 

Theater  Support  Command 

Universal  Joint  Task  List 

United  Kingdom 

United  States 

United  States  Agency  for  International  Development 

United  States  Government 

United  States  Navy 

Verification  and  Validation 

Vulnerability  Assessment  and  Protection  Option 

Working  Group 

Weapons  of  Mass  Destruction 
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171035 

WMif  1 1  OpgTTDqff*  «TCB  SOQWy 

Analysis  for  Non-Traditional 
Security  Challenges: 
Methods  &  Tools 

Military  Operations  Research  Society 
(MORS)  Workshop 

Outbrief  for 

JADM-IPT 

27  February  2006 


Agenda _ 171835 

•  Workshop  Overview 

•  Plenary  Insights 

•  WG  Selected  Findings  and 
Recommendations 

•  Common  Themes  and  Summary 
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Workshop  Objectives  mms 

NillUrr  OpfTV  tKSTV  KcmujxIi  Sgcfegfr 

•  Illuminate  users  of  non-traditional  modeling 
and  simulation  gaps  and  deficiencies  in  the 
context  of  the  defense  security  challenges  and 
emerging  changes  in  the  defense  strategy 

•  Identify  areas  of  user  commonality 

•  Capitalize  on  Joint,  Service  and  Agency 
toolsets  and  ongoing  model  development  for 
the  larger  community 

•  Develop  partnerships  to  share  future  M&S 


Workshop  Guidance 

MUttary  OtnitKHil  kMfeth  &>CH*Y 

•  Get  the  right  match  of  stakeholders, 
technologists  and  thinkers  into  each  WG 

•  Encourage  participation  by  innovative  people 
with  new  ideas  and  new  tools 

•  Shape  the  meeting  toward  the  goal  of  investing 
in  common  usable  products  to  advance  our 
ability  to  aid  decision  making 
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Attendee  Demographics  lilSRS 

Hbfbrr  Opor^bww  HcUmrch  5oO*tr 


l 

e- 


■  Comrcrcta]  p/S 

■  US  Mfeary  Rrp 

■  US  MJury  Rep  (Civjfan) 

□  Cotrmercu]  EdRDC 

■  FFRDC 

■  Oihcr  US  Govt 

■  OSD 

0  Alhed  Govcrrvnexl 

■  Commercial  Mlj; 

□  Stiff 

®  AcKknic 
S  Edmtryr. 


Total  Attendance:  183 


Speaker  Perspectives  From  the  Plenary 

Oumtaii  Will  Ell Swat* 

•  Keynote  -  VADM  Chanik 

-  We  are  re-balancing  future  force  capabilities 

-  “Fighting  the  Long  Wad*  focuses  on  key  points  that  have  not  been  addressed  in 
previous  traditional  warfare  analysis  —  building  relationships  and  early  preventive 
measures).  How  these  are  addressed  in  M&S  tools/methods  is  a  critical  question 

-  How  can  desired  QOR  end  states  be  evaluated  using  traditional  tools  and 
methods?  How  differently  are  these  non-traditional  warfare  questions  to  previous 
traditional  warfare  questions? 

•  Cyber  Warfare  -  Dr  Kass 

-  Cyberspace  is  the  only  facet/environment  that  the  US  has  peer  competitors 

-  It  ts  the  only  entirely  manmade  environment  that  we  operate  in 

•  Applicability  of  OA  techniques  -  Dr.  Hanley 

-  Traditional  OA  techniques  do  apply  to  the  non  traditional  challenges 

■  Multidisciplinary  teams 

•  Analytic  Methods:  Operational  Statistics,  Operational  Experiments,  Analytic 
Wargaming 

■  “Hemibef  thinking"  -  Factor  of  3  improvement 
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Service  Perspectives  from  the  Plenary  lilGH5 

MJIHary  OpuHtmi  fmavch  Society 

*  Air  Force 

-  The  Non -Traditional  Security  Chaifenge  requires  us  to  think  of  our  enemies  and  ourselves  as  a  part 
of  a  complex  system  —  with  many  interdependent  pans 

-  To  analyze  a  complex  system,  we  must  first  understand  its  behavior  —  Not  only  behavior  of  the 
parts  .but  how  they  form  the  behavior  of  the  whole 

*  Navy 

-  What  methods,  techniques  and  tools  can  be  applied  to  answer  questions  assorted  with  n on- 
traditional  challenges? 

-  How  do  we  get  trusted  results  that  decision  makers  are  willing  to  use7 

-  Where  we  do  not  have  accepted  methods  and  tools,  how  do  we  fili  the  gaps? 

*  Marine  Corps 

-  The  inability  to  assess  the  interdependency  of  military,  civilian,  and  multi-national  efforts  during  the 
conduct  Of  stability  operations  Is  the  Mayor  Analytic  Shortcoming 

-  Success  needs  to  be  dearly  defined  and  translated  into  quantitative  MGEs  before  analysts  can 
build  models 

*  Army 

-  Strategy  Mix  and  integration 

*  Scenario  Mix;  Force  Management;  Force  Planning,  Acquisition;  Stationing  and  Integrated 
Global  Posture  and  Basing 

■  Each  area  requires  different  analysis  techniques 

-  Common  themes:  complex  environment,  non  combatants,  C4ISR 


Perspectives  From  the  Plenary  m ms 


*  Common  Themes/Additional  Insights 

-  This  is  not  a  new  problem  but  still  not  well 
understood 

-  Need  to  understand  the  problem  before  building 
the  model 
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WG  1  ■  GWOT/Irregular  FllCiHS 

HU(tafT  o trnmtow*  tUMHFtrh  5*™-tv 

•  Complex  topic 

•  Focused  on 

—  Action  ->  Effect  ->  Results 

-  Hard  problems  involving 

■  Human  factors,  media,  influence,  populations 

■  DIME/PMESII 

■  Analysis  in  support  of  Headquarters,  Combatants,  not  training 
etc. 

-  Understanding  the  problem  and  existing  techniques 

-  Identifying  new  approaches  that  would  work 


WG  1  -  It’s  a  Process 

mens 

•  The  question 

•  The  data 

•  The  analysis 

•  The  tools 

*  New  ideas 

*  Recommendations 
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WG  1  -  Get  the  Question  Right  I71GH5 

mwin  DpnHHi  Miwrt  Sv^itif 

*  Goals  and  questions  are  often  unclear 

-We  may  lack  basic  frameworks  for  understanding  the 
problem 

•  Use  frameworks  where  they  exist 

*  You  can  model  anything,  but  it  may  ultimately 

not  be  very  helpful 

*  The  question  is  important 

-Because  soft  issue  models  often  need  to  be  customized 
for  the  question 

-And  data  may  not  exist,  or  may  be  complicated  and 
difficult  to  collect 


WG  1  -  We  Need  Data  lilSHS 

•  Lack  of  data  poses  a  significant  problem 

•  Discussed  polling  as  an  example 

-Legal  constraints 
-  Political  constraints 
-Methodological  constraints 

•  Much  of  the  data  we  need  involves  “human” 
data  that  is  difficult,  messy,  and  controversial 
to  collect 

•  Simulations  can  only  go  so  far  when  compared 
to  real  world  data 
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WG  1  -  The  Analysis  _ QV8H5 


•  Operations  research  is  the  application  of  scientific 
methods  to  operational  problems 

-  We  may  have  to  admit  that  some  things  do  not  fall  into  the  category 
of  physical  science 

—  This  means  managing  the  risk  associated  with  decisions  in  new 
ways 

•  We  may  want  to  explore 

-  New  decision  making  models  and  processes 

-  Tools  that  may  not  be  capable  of  being  validated 

-  New  techniques,  such  as  multiple  instances  of  models  and  theories 


WG  1  -  The  Tools 

|i!G^5 

•  No  new  or  revolutionary  tools 

•  Considerable  skepticism  about  any  one 
model’s  ability  to  solve  a  problem 

•  All  the  existing  tools  tend  to  be 

-  Specialized 

-  Expert  intensive 
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WG  1  -  Existing  Models  n\ms 
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WG  1  -  Unconventional  Tools  lilSRS 
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WG  1  -  New  Stuff 

171 SR5 

*  Games  as  tools 

-  What  makes  a  game  helpful? 

-  Scenario-based  planning,  seminar  wargames,  MMOGs? 

•  Learn  from  other  communities 

-  Computational  social  science 

-  Commercial 

■  Interface 

•  Usability;  flexibility,  agility 

*  Focus 

•  Data  mining  tools 

—  Figure  out  what  they're  good  for 

WG  1  -  What  Needs  to  be  Done  H10H5 

NUtarr  opnlto,  ^MarcH  Schl^u 


•  Frameworks/paradigms  for  Irregular  Warfare  Analysis 

•  Irregular  Warfare  M&S  Evaluation  Criteria 

•  Test  Cases 

•  Irregular  Warfare  Scenarios 

•  MOEs 

•  Data 

•  Irregular  Warfare  M&S  at  the  Campaign  Level 

•  W&A 

•  Analyst  (and  Consumer?)  training 

•  Prototype  cross-cutting  studies 

•  Best  of  breed  approach  to  model  improvement 


Appendix  B  -  9 


WG  1  -  A  Few  Suggestions  li1©H5 

WtHT  OpvilKm  *«*Mi-h  Society 

•  Change  in  the  consumption  of  the  analysis  may  be  the 
“killer  app” 

-  Develop  ways  to  deal  with  uncertainty  in  the  inputs  and  results 

—  Develop  ways  to  account  for  and  balance  risk  involved  in  different 
experimental  approaches  and  outcomes 

•  Consider  expanding  the  role  of  wargames  as  tools  that 
support  decision  making 

-  But  they  have  to  be  done  right 

-  And  you  have  to  balance  them  with  models 

•  Learn  to  use  range  of  outcomes  provided  by  multiple 
independent  analytical  approaches 


WG  1  -  Recommendations  m©RS 

•  Getting  the  question  and  the  framework  right  is 
important  -  and  relatively  inexpensive 

-But  it  can  take  time,  and  iteration 

•  Examine  the  possibility  of  collecting  real-world 
data  to  support  decisions 

■  Examine  groups  of  models 

•  Research  the  use  of  games/scenario  planning/ 
commercial  techniques  to  support  decision 
making 
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WG  1  -  More  Recommendations  IT16R5 

MDttarr  OpntHHH  lhnn.il  i«Mf 

•  Examine  new  ways  to  validate  models,  and  find  ways  to 
deal  with  what  traditionally  cannot  be  validated 

-  Any  model  that  deals  with  human  decision  making 

-  While  they  can  not  be  validated  they  can  be  used  to  inform  and 
support  decision  making 

-  Traditional  assumptions  of  reproducibility  and  traceability  may  need 
to  be  replaced  with  transparency  and  acceptance  of  judgment 

•  Examine  thinking  about  the  role  of  modeling,  and  the 
need  for  scientific  answers  for  questions  that  may  have 
no  basis  in  physical  science 

-  May  need  to  accept  more  risk  in  this  area 


WG  3  -  Catastrophic 


IilS35 

Hiltai  t  Oemoom  1—11)1  inemr 


•  Catastrophic  Homeland  Events:  Man-made  or  naturally  occurring 
events  involving  significant  loss  of  life  and  significant  social/political/ 
economic/psychological  effects;  e.g.  large  National  Planning  Scenarios 
(10KT  nuke,  chlorine  tank,  plague,  anthrax,  large  natural  disasters) 

-  C01:  NORTHCOM,  PACOM,  Services,  DTRA,  National  Labs,  DHS 
(including  Coast  Guard,  FEMA),  HSC,  EPA,  DOE 

•  Catastrophic  Counter-WMD  Events:  WMD  interdiction,  counter- 
proliferation,  deter/defeat  the  use  of  WMD  by  state  and  non -state  actors 

-  COI:  STRATCOM,  Services,  Regional  COCOMS,  DTRA,  National  Labs 
DOE 

•  Significance  =  f  (physical,  economic,  social,  political  and  psychological 
effects,  level  of  effort,  duration) 

•  Some  key  unresolved  issues 

-  Rotes  and  relationships  (Lead  Federal  Agency,  Title  10/32,  authority) 

-  Lack  of  coordination,  integration,  inter-operability 

-  Immature  data,  methods,  tools  in  some  areas 
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WG  3  —  Current  Analytic  and  M&S 


li1@R5 
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•  Current  capabilities 

-  Engineering  level:  Robust  individual  models:  DTRA  Suite,  CAPS 

-  Tactical/operational  level:  EPICS,  JCATS,  IWMDT 

-  Campaign  level:  Legacy  models  (ITEM),  but  these  are  not  tailored  for 
domestic  catastrophic  events 

-  Training  models:  Significant  lack  of  simuiators/stimulators  and  facilities 

•  Applications/examples  -  many  stand  alone  tools:  multiple 
tools  often  required  to  assess  a  scenario 

-  Defense  Collaboration  Tool  Suite/InfoWorkSpace  (DCTS/IWS) 

-  Hazard  Prediction  Assessment  Capability  (HPAC) 

-  Consequence  Assessment  Toolset  (CATS) 

-  Integrated  Munitions  Effects  Assessment  (IMEA) 

-  Vulnerability  Assessment  and  Protection  Option  (VAPO) 

-  Traditional  campaign  analysis/mission  level  tools  re-purposed  to 
assess  catastrophic  scenarios 


WG  3  -  On-going  Enhancements  ms35 

Uttar*  Dpntkm  Km  ■re*  Scotty 

*  Selected  engineering  and  system  level  tools 

-  Joint  Effects  Model  (JEM):  Predict  and  track  NBC)  and  Toxic 
Industrial  Chemical/Material  (TIC/TIM)  events  and  effects 

-  Joint  Operational  Effects  Federation  (JOEF):  M&S  for  warfighters 
and  planners  to  accurately  predict  CBRNE  effects 

-  The  Joint  Warning  and  Reporting  Network  (JWARN): 

Comprehensive  analysis  and  response  capability  to  minimize  the 
effects  of  NBC  attacks,  accidents  and  incidents 


DTRA  will  begin  to  spiral  Integrated  WMD  Toolset  (IWMDT)  FY06 


*  DTRA  Collaboration  Center  (DCC)  -  Common  architecture  that 
supports  CBRNE  data  fusion  from  disparate  sources  supporting 
Situational  Awareness,  Systems  Engineering  and  capability 
development:  initial  operational  capability  FY06 


DTRA  partnering  with  Services,  COCOMS,  Allies,  Labs 
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WG  3  —  Analysis  Gaps  and  Deficiencies 

^ HUItwY  Operitmnt  Rawdi  Society 

•  Key  stakeholders  and  decision  makers 

-  USG  {Inter-agency)  ♦  DoD/Services/COCOMS  410081  governments 

-  Allies  ♦Coalitions  ♦Industry 

-  State/territorial  Govts/NG  ♦  Nations  I /private  labs  ♦  First  responders 

•  Key  analysis  gaps  and  deficiencies 

-  Perceived  lack  of  objective  goals,  defined  responsibilities,  actions/resources  reflecting 
stated  priorities 

-  Need  for  improved  interagency  cooperation,  collaboration  and  data  sharing 

*  Insufficient  understanding  of  complex,  primary  effects  and  secondary  effects  (political, 
economic,  social,  psychological) 

“  Little  quantification  of  local,  state,  federal,  DoD,  and  international  response  capabilities 

-  Immature  and/or  non-existent  operating  procedures,  data,  tools  for  evolving  mission  and 
requirements 

-  Barriers  to  sharing  data,  plans  (classification  issues,  terminology,  ontology) 

•  Commonalities  between  stakeholders 

-  Desire  for  comprehensive,  systematic,  interagency  analysis 

-  Desire  for  integrated,  tactical/operational  tools 

-  Need  to  plan  for  response  and  future  investments 

-  Mission  focus 


WG  3  -  Key  M&S  Shortfalls  and  Gaps 

■  Data 

—  Lack  of  data  on  local,  state,  other  federal,  NGO,  international  agency  capability 

-  Biological  threats,  retransmission,  mortality,  etc. 

-  CBRN  effects  (lack  of  current/empirical  test  data) 

-  CONOPS 

-  Desired  response  over  time 

-  Metrics 

-  Translation  of  DoD  capabilities  to  HLD  employment 

■  Pre-event  (detect,  deter,  prevent  capabilities) 

■  Post-event  (consequence  management) 

-  Apportionment  of  tasks  across  services 

*  Methods/Tools 

-  Model  integration/synergy  (output  of  one  model  not  always  useable  by  others  to 
provide  useful/meaningful  output) 

-  Catastrophic  models  (e.g.  Biological) 

-  Empirical  based  models 

-  Sodo-economtcal,  psychological,  and  political  aspects 

-  Integrated  M&S  “pyramid,"  particularly  at  higher  levels 
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WG  3  -  Potential  Solutions  lilGftS 

HiMurr  op ntkra  ivwt h  swctv 

-  Data/CONOPS 

-  DHS  development  of  a  database  of  state  and  local  capabilities  (Target  Capabilities  List) 

-  Establish  proponents  for  interagency  data  collection  for  engineering  to  campaign  level 
analyses, 

-  Deveiopment/assessment  of  appropriate  Analytic  Agenda  scenarios  and  baselines 

-  Leverage  JFCOM  Experimentation,  Joint  Training,  and  Joint  Center  for  Operational  Analysis 
events/processes  for 

■  Development  of  Combating  WMDrCatasIrophic"  analytical  toois/methods 

■  Relevant  data  (e.g.r  Joint  Rapid  Distributed  Database  Development  -  JRD3) 

■  Potential  M&S /analytical  solutions/needs  derived  through  experimentation 

-  Methods/Tools 

-  Facilitated  war  games  (EPiCS  -  Emergency  Preparedness  Incident  Command  Simulation. 
JSRTS  -  Joint  State  Response  Training  Simulation,  TOPOFF  Ex) 

-  DTRA's  fielding  of  an  integrated  GtG-accessible  suite  of  CBRNE  tools, 

-  Enhancements  to  campaign  models  (ITEM  to  link  1WMDT) 

-  Commercial  game  adaptations/gaming  engines  (e.g.  SimCity,  De!ta3D) 

-  Potential  partnerships  for  new  M&S/data  development:  DTRA,  DHS  (including  Coast  Guard, 
FEMA),  National  Guard  Bureau,  JFCOM.  NORTHCOM,  STRATCOM,  UK,  Canada.  Service 
labs  (AFRL),  industry. 


WG  3  -  High  Priority  Analytic  M&S  Needs  HIGHS 

NJlHwv  DfMntMnn  ■  1 1  i  ;b  fee l«r 


Focus  on  data  and  establishing  an  M&S  Community  of  Interest 


*  Data 

-  Establish  a  list  of  questions  for  catastrophic  events  requiring  analysis  (esp  COCOM  needs) 

-  Collect  exercise  and  operational  data;  Unit  commitment,  deployment  timeline,  activities,  duration  of 
commitment 

-  Collect  war  game  results:  02  relationships,  roles,  CONORS,  metrics,  need  for  DoD  forces,  Title  10/32 
relationships 

-  Begin  to  re-validate  applicability  of  basic  engineering  data  (lab  testing  for  CBRNE) 

-  Develop  biological  warfare/ contagious  disease  data:  Retransmission,  morbidity,  etc 

-  Follow  up  on  DHS  efforts  to  develop  systematic,  comprehensive  development  of  data  on  local/ state 
response  capability 

-  Support  common  data  standards  (metadata) 

*  Methods 

-  Increased  catastrophic  scenario  involvement  in  analytical  agenda  products 

-  Support  the  conduct  of  facilitated  war  games  to  develop  CONORS,  concepts,  metrics  and  clarify 
responsibilities 

-  Support  SPG  tasker  to  assess  large  consequence  management  events 

*  Tools 

-  Assess  DTRA  Collaboration  Center  ability  to  support  analytical  requirements 

-  Establish  an  interagency *  *HLD  M&S  Community  of  Interest"  to  refine  requirements  and  integrate  effort 
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WG  3  -  Recommendations 

Hlltert  Qrtrtbw  ftwwcti  Socier* 

*  Specific  to  WG  area 

-  Develo  p  data  bases  a  n  d  sou  nces  for  tha  t  d  ata 

-  Conduct  analytically  supported  exercises,  wargames  and  studies  to  refine  CONORS,  metrics 

-  Develop  an  active  and  robust  interagency  M&S  Community  of  Interest 

-  Consider  redefining  “catastrophic'  as  two  missions  areas 

*  Pre-event  (prevent,  deter,  defeat) 

*  Post-event  (consequence  management) 

*  Overall  to  Non-Traditionaf  Challenges 

-  Improve  communications  between  all  stakeholders  by  increased  focus  on  Non-tradilional 
challenges  in  MORS.  JCAC.  and  other  joint  and  inter-agency  forums 

-  Develop  M&S  Communities  of  Interest  for  each  of  the  sub-areas 

-  Reapportion  M&S  funding  from  traditional  to  norvtraditional  models 

-  Renew  emphasis  on  analytically  supported  war  gaming 

-  Refocus  the  analytic  agenda  on  non-tradftional  scenario  development  and  analysis 

-  Date-mine  DTOs.  goals,  objectives,  and  funding  profiles 

*  Other  comments 

*  Need  to  support  inter-agency  participation 

-  Neecfto  recognize  that  significant  policy  and  legal  issues  remain  unresolved 

-  H  the  next  &-1 1  or  OIF  stability  operation  happens,  will  we  be  able  to  say  wa  ve  done  our  best  to 
plan  and  prepare? 


WG  4  -  Focus  Questions 


mcas 


•  How  can  analysts  characterize  and  measure 
uncertainty  concerning  deterrence? 

•  How  can  deterrence  analysts  determine  and 
manage  second-  and  third-order  effects  in 
deterrence  analysis? 

•  How  can  analysts  estimate  the  magnitude  of 
an  adversary’s  response  to  a  proposed  friendly 
deterrent  action?  Methods  exist  to  determine 
the  general  response  (e.g.,  favorable, 
unfavorable).  This  question  concerns:  “How 
favorable?"  or  “How  unfavorable?" 
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WG  4  -  Structure  Hl©35 

•  Stage  Setting 

-  DoD  Strategic  Deterrence  Concept 

-  Deterrence  Assessment  Framework 

•  Problem/Discussion 

-  Effects  in  the  Cognitive  Domain 

Discussion:  Measuring  Uncertainty 

-  Understanding  Chinese  Thinking 

-  Gauging  Adversary  Response 

Discussion:  Adversary  Response 

-  Planning  and  Integrating  Deterrence 

Discussion:  2nd  and  3rd  Order  Effects 

•  Synthesis 

-  Problem  Description 

•  Methods  and  Tool  Availability 

-  Recommendations 

WG  4-  Defining  Deterrence 

megs 

•  Scope  of  deterrence  today 

Definitions 

-  Ends:  Beyond  the  most  dire  threats 

are  important. 

-  Ways:  Beyond  coercion  to  include  inducements 

-  Means:  Beyond  military  to  include  nommiiitary  (inter-agency) 

*  Specifying  the  problem(s) 

-  Coercion 

•  Basic,  extended,  intra-war,  competence 

-  Inducement 

8  Engagement  (alliances,  coalitions),  cooperation  (international 

organizations),  assurance 

*  inclusion  of  inducements  expands  deterrence  into  the 

realm  of  influence 

-  MOEs  and  analysis  for  each  of  these  may  different 

-  Perception  by  others  may  be  different  {assume  deterrence  is 

coercion) 

Appendix  B  -  16 


Government  Stakeholders  &  Roles  in 

Developing  Deterrence  Policy  &  Plans 


Iil635 
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*  USG:  NSC,  Inter-Agency,  ICt  Congress 

-  National  objectives:  National  Security  Strategy* 

-  Develop  policy 

-  Prioritization  and  allocation  of  resources 

*  DoP 

-  Military  objectives;  National  Military  Strategy 

-  Develop  policy 

-  Issue  guidance;  TSC  guidance,  CPG,  SPG 

-  Prioritization  and  allocation  of  resources 

*  STRATCOM 

-  Develop  global  deterrence  plan 

-  Synchronize  deferrence  planning  in  other  plans 

*  Other  COCOMs 

—  Regional  Commands,  including  NORTHCOM;  Develop  regional  deterrence 
plans 

-  SQCQM:  Develop  transnational  deterrence  plans 


Roles  are 
important. 


WG  4  -  Deterrence 


m©ns 
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•  Workshop  Insights  for  the  Global  Deterrence 
Concept 

-  Definition 

-  Scope 

-  Cyber-deterrence 

-  Non -state  actors 

-  Communication  of  deterrence  actions 

-  Implementation  and  planning 

-  Costs  of  deterrence  to  the  US 
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WG  4  -  Deterrence 

Attacking  the  Deterrence  Analysis  Problem 


Dimensions/factors 

^Players  (includes  capabilities  and 
limitations) 

-Dedsions/actions 
-Competition/cooperation 
-Determi  nistic/stocha  Stic 
-Goals/objectives  (single  v  multiple. 
Political,  econ,  mil?) 

-Payoffs  (+/-,  Costs,  Benefits)  reduce  prob 
of  achieve,  reduce 
bensfits/in  crease  costs, 
preference  ordering 
■Stage  s/phases 

-Strategies  (risk  averse,  mini-max,  maxi- 
min) 

-Interaction  ^Influences  (2nd,  3rd  order 
effects) 

-Others,..,* 


Match  withu 

III 


QR  Methodologies  (Suite) 

-Dynamic  programming  (stochastic  or 
deterministic) 

-Game  theory  -  {>  2  person),  (non  0-sum ), 
{multi-stage) 

-Experimental  gaming 
-Influence  diagrams 
-Cost-benefits  analyses 
-Risk  analysis  (p[event]  *  consequence) 
-Decision  trees/decision  analysis  (Value, 
Utility) 

-Cognitive  mapping  Difference 

Today 

-  Who 
What 

-  How 

-  Context 


Application  of  Methodology  depends  on  the  dimensions  AND  the  question 
being  asked  or  the  decision  to  be  made! 


Appropriate  question:  Do  certain  actions  contribute  (detract  from)  deterrence  or  stability? 
Bad  question:  What  is  the  probability  that  action  X  will  deter  actor  V  from  doing  Z? 
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OR  Tool  Availability 
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♦  Significant  Gaps  in  measuring  effects,  determining 
the  impacts  of  second  and  third  order  effects,  and 
accounting  for  uncertainty, 

♦  Data  availability  eliminates  some  models. 

♦  Timely  methods  required  to  inform  decisions. 

♦  Human-in-the-ioop  methods  often  best  choice. 


yorrwiainp 
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WG  4  -  Points  to  Consider 

moH5 

*  Problem  Definition 

-  Problem  sufficiently  scoped;  work  needed  to  bound  the  problem  (All 
deterrence,  or  just  strategic  deterrence?  All  forms  of  influence?) 

-  How  do  we  account  for  the  dynamic  nature  of  the  problem?  (. .  .the 
“wicked"  nature,..) 

*  Metrics 

-  Are  we  asking  the  right  questions  to  guide  metrics  development? 
(What  are  the  metrics?) 

-  What  are  the  questions  that  should  drive  the  development  of 
metrics? 

*  Data 

-  Lack  of  relevant  data  drives  need  for  new  methods  to  impact 
decisions  in  a  timely  manner 

-  Some  tools  should  work  given  unrealistic  data  requirements 
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WG  4  -  More  Points  to  Consider  I7ISR5 
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*  Methods 

-  Allow  for  competitive  analysis;  value  of  outsourcing  analysis; 
avoiding  single  points  of  failure 

-  Impact  of  SME  and  expert  opinion  as  data  source 

-  Need  for  inter-disciplinary  team  (include  IA,  multi-national) 

-  Need  for  methods  to  validate  analysis 

-  Methods  to  address  the  role  of  ambiguity  in  deterrence 

-  From  plenary:  Methods  must  be  usable,  understandable, 
extendable,  and  replicable 

-  How  do  we  account  for  learning  by  unidentified  (i.e.,  potential) 
adversaries? 

•  MORS 

-  Provide  mentor  to  work  with  each  WG 

-  Accommodate  and  seek-out  IA  participation 


WG  4  -  Recommendations  IT1GR5 
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•  Recommendations 

-  To  Deterrence  WG/STRATCOM 

■  Implement  suggestions  throughout  this  brief 

-  To  Sponsors 

■  DoD:  Carefully  define  and  specify  scope  and  roles  concerning 
deterrence 

■  Drive  development  of  deterrence  policy,  analysis  and  planning  across 
USG,  not  just  within  DoD 

■  Sponsor  focused  (and  integrated?)  research  in  difficult  areas 

-  To  MORS 

*  Provide  a  mentor  to  each  WG  (ala  Rist  mentors) 

•  Adjust  MORS  to  accommodate/encourage  greater  I A  participation 

■  Hold  MORS  Workshop  on  Influence  Operations 

*  Will  help  define  boundanes 

■  Sponsor  MORS  competition  for  thorny  problems 
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WG  5  -  Decision  Making  and  Integration 


filSHS 

Wary  Operation*  tooety 


Ai 

I  ti¬ 


ck 5^jjfj§f**!EL‘  Dtfpanmoi  n  ituftji 


,  capablhrio*  cc-  add  nets  i  rrogtiinr! 

caianroptoc£mi  dJstupthrc  chpllcrt*?'.  whijeiuftamim; 
cnpitvljirtc*  to  addra*  tradkional  dnuallcnj*^ 

QDR,  page  1 9 


irregular  thtf  !k'frg<?&  *  Catastrophic  Challenges 


Traditional  Challenge  Dltryptivo  Challenges 


*  Framework  for  Analysis  for 
each  of  4  QDR  focus  areas 

-  Characterize  future  (2025) 

■  Range  of  operations  that  our 
forces  may  be  required  to 
support 

->  Identify  decision  making  and 
integration  questions 

-  Assess  QDR  capabilities 

*  identify  capabilities  that  will 
infiuencarfo nee  sizing 

•  Identify  or  develop  measures  for 
force  sizing  capabilities 

•  How  would  this  change  the 
joint  analytic  agenda 

-Tims  phase 


WG  5  -  Conclusions  IH©7?S 

WUttnr  Opera  t>w»  Fi.n***r? h  Smmtt 

•  The  future  makes  scenario  based  planning  more  important 

—  Uncertain  and  dangerous 

-  Complex,  intelligent,  adaptive  threats 

•  Larger,  more  diverse  group  of  decision  makers  and  stakeholders 

-  Difficult  to  trade-off  between  diverse  capabilities 

-  Fundamental  issue  may  be  balancing  risk 

•  Non-traditional  complexity  presents  new  integration  challenges 

-  Many  complexities  caused  by  threats,  distributed  decision  making,  and  very 
large  number  of  stakeholders 

-  Difficult  to  integrate  priorities  across  non-traditional  security  areas 

•  Defining  functions  of  each  focus  areas  will  help  measure,  model 
and  simulate  QDR  capabilities 

-  We  need  measures  for  some  of  the  most  important  capabilities 

Makes  Title  10  provider’s  job  more  challenging. 
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WG  5  -  Recommendations  and 
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■  Caveat  -  WG  5  did  not  examine  specific  models  and  simulations. 


-  New  focus  on  risk  analysis  methods 


Next  steps 


-  For  each  QDR  focus  area  and  across  the  focus 
areas 

•  Use  creative  new  scenarios 

■  Define  functions 

•  Identify  capabilities  to  perform  functions 

•  Develop  measures  for  each  capability 

■  Develop  models  and  simulations  to  calculate  measures 


The  Problem  is  Not  New 


*  But  it  does  reflect  a  shift 

irregular 

tin* 

iHHlB 

in  the  Department's 

priorities 
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*  What  is  “non-traditionar  in  the  non-traditionaJ  challenges? 

-  Interdependence  and  relationships  with  other  non-defense 
organizations 

-  Link  between  methodology,  metrics  and  decision  makers 

-  A  complex  system  with  many  interdependent  parts,  influenced 
by  many  factors  -  ours  and  theirs 

-  Importance  of  perception 

-  Less  data  available  (increased  use  of  social  science  methods) 

-  Some  capabilities  can  be  difficult  to  measure 
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The  Working  Groups:  An  Assessment  mens 

*  Did  we  get  we  get  the  right  match  of  stakeholders, 
technologists  and  thinkers  into  each  WG? 

-  Overall  workshop  had  a  diverse  group 

-  With  diversity  comes  more  time  reaching  a  common  frame  of 
reference 

—  Involving  other  government  organizations  and  agencies  is  still  a 
challenge 

*  Working  Group  pre-workshop  preparation 

—  Catastrophic  WG:  Stakeholder  survey’s  to  frame  discussion 

-  Deterrence  WG:  Low  response  rate  a  problem 

—  Irregular  WG:  Innovative  use  of  scenarios  to  focus  discussion 


The  Working  Groups:  Common  themes 

HWt*rv  OpmttoM  BwiA  souttv 

*  Scoping  DoD  mission 

-  is  some  cases  the  DoD  is  the  supporting  organization 

-  Defining  capabilities  expected  from  the  DoD 

*  What  are  decision  makers  key  issues  and  what  are  the  suitable 
measures? 

*  Defining  interaction  with  other  governments,  NGOs  and 
international  organizations 

-  Roles  and  responsibilities 

-  Different  organizational  structures  and  cultures 
—  Information/data  providers 

*  Methods 

-  Analytic  wargaming 

-  Non-M&S  methods  (risk  analysis,  cost-benefit  analyst,  social  networks, 
game  theory 

-  Integration  across  an  ensemble  of  models/tools 

_ 
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The  Working  Groups:  Common  themes  I71GR5 

HHitirr  OpentHm  RH*utfi  *xi«f 

*  People 

-  Need  more  diverse  analytic  teams  with  more  social  science 
experience 

*  Data 

-  Where  is  it?  How  to  acquire?  What  is  available? 

-  Interaction  between  data,  models  and  metrics 

*  V&V 

-  How  to  accomplish  V&V  for  new  tools 

-  When  evaluating  a  model,  at  least  two  broad  standards  are 
relevant,  One  is  whether  the  model  is  consistent  with  the  data.  The 
other  is  whether  the  model  is  consistent  with  the  real  world.  - 
Kenneth  A.  Bollen,  Structural  Equations  with  Latent  Variables 


Scorecard 


mens 


•  Accomplishments 

-  The  Workshop  made  progress  against  these  objectives 

■  Review  of  current  tools 

*  Identify  gaps  in  skill  sets  and  requirement  for  more  diverse  OR  teams 

-  To  paraphrase  the  psychiatrist  in  Portnoy’s  Complaint  "Now,  we  are 
ready  to  begin,.." 

*  Residual  Challenges 

-  Reach  out  to  a  broader  community  {e.g.,  Inter-agency,  coalition) 
continue  to  broaden  open  and  collaborative  environments 

-  Regard  this  workshop  as  part  of  a  sequence,  continue  the  work  started 
In  the  past  workshops  and  the  analytical  methodology  working  group 

-  Continue  to  develop  analytical  methodologies  to  analyze  non-traditional 
challenges 

-  Methodologies  to  facilitate  assessing  and  balancing  risk 

-  W&A  should  address  methodologies  and  data  as  well  as  tools 
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The  Workshop  Message 


mens 

WWarf  Opwntttn*  (jwtH  seOrty 


•  While  there  is  some  new  tool  development 
ongoing,  we  are  really  in  the  question 
definition,  data  gathering  and  methodology 
development  phase  for  Non-Traditional 
Analysis 

•  We  need  to  open  the  lens  and  reach  out  to 
other  disciplines  and  involve  non-DoD 
agencies  in  the  process 


The  Road  Ahead 

meas 

*  Opportunities  Identified  for  Sponsors 

*  JADM  SC  Outbrief  2  March 

*  Follow-on  Navy  APL  Task 

*  STRATCOM  Deterrence  JOC  rewrite 
■  March  IDA  Irregular  Workshop 

*  March  Analysts  M&S  Master  Plan  Workshop 

*  DTRA  WMD  Workshop 

•  DRTA  technical  paper  on  combating  WMD  for  PHALANX  to  describe  new 
missions  and  required  supporting  analysis 

•  March  Loose  Nuke  MSFD  Conference  Methodology  WG 
■  April  Tholian  WES  Wargame 

•  April  CBP  11  MORS  Workshop 

•  June  74th  MORSS 

-  Special  Session  Workshop  Briefout 

-  WG  papers  (classified  &  unclassified) 

*  Need  for  Follow -on  Workshop??? 

_ 
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Analysis  for  Non-Traditional  Security 
Challenges: 

Methods  &  Tools 

Workshop  Out  Brief  For 
JADM  Steering  Committee 
_ 6  April  2006 _ 

Sunny  Conweli 
N816  WCM  Lead 


MORS  Workshop:  “Analysis  for 
Non-Traditional  Security  Challenges" 
21-23  February  2006 


•  Co-sponsored  by  US  Navy  (N81)  and  US  Air  Force  (HQ/A9)  at 
the  Johns  Hopkins  University/Applied  Physics  Lab 

•  Participants:  Representatives  from  OSD,  Joint  Staff, 
Services,  COCOMs,  allied  nations,  industry,  academic 
groups 

•  Forum:  Community  review  of  tools,  technologies,  and 
methodologies  that  enable  the  analysis  of  non-traditional 
warfare 

•  Purpose:  Match  stakeholder  analytic  gaps  with  potential 
model  or  tool  solutions 

•  Goal:  Improve  DoD  models  and  tools  used  to  support 
analysis  of  irregular,  catastrophic,  and  disruptive  threats 
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Workshop  Leadership 


HUltvr  OpvwtK-n  fiew-n* *  Society 


*  Dr.  Jacqueline  Henningsien,  FS,  USAF 
-  Mr.  A.H.  Barber  lllr  FS,  USN 


*  Mr.  Lee  Dick,  FS,  USN  NB1 
■  Dr.  Jim  Harris.  USAF  A9 


-  Keynote:  VADM  Chanik  (DJB) 

*  Cyber  Warfare;  Dr.  Kass  (C3AF) 

*  OA  Techniques:  Dr.  Hanley  (IDA) 

-  Service  and  OSD  Gaps  and  Needs: 

-Mr.  Barber!,  USN  N81 
■Dr.  Hennlngsenf  FS,  USAF  A9 
■Dr.  Markowitz,  USA  CAA 
•Dr.  Akst,  USMC  MCCDC 
■COL  Pat  Kelly.  OSD-F 


Co-Chairs: 

Dr.  Ed  Me  Grady.  CNA 
Mr.  Mike  OnenberQ 
OSD-PAE/ATL 


Co -Chairs: 

Dr.  Al  Swaetser.  OSD-PA&E 
Mr.  Walt  Zimmers,  DTRA 
Mr.  Don  Clements,  OSD-PA&E 


Co-Chairs: 

LTC  Steve  Reis#, 
STRATCOM 
Mr,  Mark  Reid,  MITRE 


Co-Chair#: 

Mr,  Roy  Reiss,  HCFA* 
Me.  Sunny  ConwelL,  N81 


WG  1  -  Irregular/GWOT 


n\ms 

HUItety  Qpntkm  Bn  [I  Sod-rty 


*  Getting  the  question  and  the  framework  right  is  extremely  important  —  and 
relatively  inexpensive 

*  Examine  the  possibility  of  collecting  real-world  data  to  support  decisions 

*  Research  the  use  of  games/scenan'o  planning/commercial  techniques  to 
support  decision  making 

*  Examine  new  ways  to  validate  models,  and  find  ways  to  deal  with  what 
traditionally  cannot  be  validated 

-  Any  mode!  that  deals  with  human  decision  making 

-  While  they  ca  not  be  validated  they  can  be  used  to  inform  and  support 
decision  making 

-  Traditional  assumptions  of  reproducibility  and  traceability  may  need  to  be 
replaced  with  transparency  and  acceptance  of  judgment 

*  Examine  thinking  about  the  role  of  modeling,  and  the  need  for  scientific 
answers  for  questions  that  may  have  no  basis  in  physical  science 

*  May  have  to  reason  across  an  ensemble  of  models 
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WG  3:  Catastrophic  _ mens 
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Consider  redefining  Catastrophic  as  two  missions  areas 
1*  Pre-Event  (Prevent,  Deter,  Defeat) 

2-  Post-Event  (Consequence  Management) 

•  Tools 

-  Assess  STRATCOM  sponsored  DTRA  Collaboration  Center  ability  to 
support  analytical  requirements 

—  Establish  an  interagency  HLD  M&S  Community  of  interest  to  refine 
requirements  and  integrate  effort 


Potential  partnerships  for  new  M&S/data  development;  DTRA,  DHS 
{including  Coast  Guard,  FEMA),  National  Guard  Bureau,  JFCONI, 
NORTHCQM,  STRATCOM,  UK,  Canada,  Service  labs  (AFRL),  Industry 


WG  4  -  Deterrence 

ms*5 

Attacking  the  Deterrence  Analysis  Problem  | 

Dimension  Sectors 

'Players  (including  capabilities  and 
[Imitations] 

-Oeqisions/actfons 


'Compel  ItioiVcoopa  rati  o  n 

-Deterni  i  ni  sti  c/stoch  astit 

'Goal ^objectives  {single  vs  multiple,  Political, 

economic,  military?) 

-Payoffs  (+A,  Costs,  Benefits)  reduce  prob 
of  achieve,  reduce 


Match  with  M., 

in 


benefits/increase  costs, 
preference  ordering 
-Stages^  phases 

'Siraiegles  (risk  averse,  mini -max,  maxi- 

min) 

-Intaractions/lnflijenoes  {2nd,  3rd  order 
affects) 

-Others . 


OR  Methodologies  fSuite) 

-Oynamc  Programming  {stochastic  or 
deterrnnisUc) 

■Game  Theory  -  {>  2  person),  (non  D-sum], 
(mulli-stBge) 

-Expen mental  Gaming 
-Influence  Diagrams 
-Cost-benefits  Analyses 
-Rtsk  Analysis  (p[even|J  ■  consequence) 
-Decision  Trees/Dedsfon  Analysis  {Value, 
Utility) 


-Cognitive  Mapping 


Difference 

Today 

-  Who 

-  Whal 

-  How 

-  Context 


Application  of  Methodology  depends  on  the  dimensions  AND  the  question 
being  asked  or  the  decision  to  be  made! 


Recommendations  To  Sponsors 

-  DoD:  Carefully  define  and  specify  scope  and  roles  concerning  deterrence 

-  Drive  development!  of  deterrence  policy,  analysts  and  planning  across  USG, 
not  just  within  DoD 

-  Sponsor  focused  (and  integrated?)  research  in  difficult  areas 


STRATCOM:  Workshop 
output  win  directly 
contribute  to 
SD  JOC  rewTilq 
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WG  5  -  Decision  Making  and  Integration 


lilSrtS 
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*  Potential  Resource  Allocation  Methods* 


-  New  focus  on  risk  analysis  methods 


Next  steps 

-  For  each  QDR  focus  area  and  across  the  focus  areas 


■  Use  creative  new  scenarios 
-  Define  functions 

*  Identify  capacities  to  perform  functions 

*  D  e  vel  op  m  easu  res  to  r  ea  ch  capa  bliity 

*  Develop  models  and  simulations  to  calculate  measures 


Working  Groups:  Common  Insights 


lilSHi 

Htttur  Open  bom  r—milH  Sodity 


*  People 

-  Need  more  diverse  analytic  teams  with  more  social  science/soft  science 
experience 

*  Data 

-  Where  is  it?  How  to  acquire?  What  is  available? 

-  Interaction  between  data,  models  and  metrics 

*  Methods 

-  Analytic  Wargaming 

-  Non-M&S  methods  (risk  analysis,  cost-benefit  analyst,  social  networks,  game  theory 

-  Reasoning  across  an  ensemble  of  models/tools 

*  V&V 

-  How  to  accomplish  V&V  for  new  tools  to  a  level  that  the  decision  maker  will 
trust  the  outcomes 


Appendix  C  -  4 


The  Workshop  Message 
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•  While  there  is  some  new  tool  development  ongoing,  we 
are  really  in  the  question  definition,  data  gathering  and 
methodology  development  stage  for  non -tradition  a  I 
analysis 


•  We  need  to  open  the  lens  and  reach  out  to  other 
disciplines  and  involve  non-DoD  agencies  in  the 
process 
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MORS  Workshop:  Analysis  for  Non-Traditional  Security 
Challenges:  Methods  and  Tools 

JHU/APL,  Laurel,  Maryland 
21  -  23  February  2006 
(Last  Updated  1  December  2005) 

TERMS  OF  REFERENCE 


1.  Background 

The  current  National  Defense  Strategy  builds  upon  the  efforts  of  the  2001 
Quadrennial  Defense  Review  and  the  dramatic  changes  in  the  security 
environment  and  lessons  learned  in  the  post  9-1 1  world.  The  strategy  identifies 
four  persistent  and  emerging  challenges  in  this  new,  more  uncertain  area.  The 
four  challenges,  traditional,  irregular,  catastrophic  and  disruptive,  are  depicted  in 
the  following  graphic. 


Security  Environment  Challenges 


po&3«  salon, 
methods  pre 


Challenges  from  tho 
"unconventional'1  metho 
the  traditional  advantages  of 
opponents. 


KQter  j 


Catastrophic 

involving  the  acquisition, 
ihd  use  of  WMD  or 

WMD-Ilke  effects. 


Lower 


Traditional 


Challenges  posed  by  states  employing 
recognized  military  capabilities  and 
forces  in  wefl-understood  forms  of 
military  competition  and  conflict. 


Higher 


Challenges  from  adv 
develop  and  use  breakthrough 
technologies  to  negate  current  US 
advantages  in  key  operational  domains. 


LIKELfHOOD 


No  hard  boundaries  between  categories,  ' 
circii instances  arise  when  we  face  a 


Figure  1 

These  persistent  and  emerging  challenges  are  further  defined  in  the  2006-201 1 
Strategic  Planning  Guidance  which  states  that  the  defense  strategy  must  be 
supported  by  capabilities  and  attributes  that  contend  effectively  with  these 
challenges.  Specifically,  components  are  explicitly  directed  to  reduce  risk  by 
developing  capabilities  to  counter  the  persistent  and  emerging  challenges. 

A  prerequisite  for  reforming  the  way  DoD  does  business  in  acquisition, 
requirements  and  trades  is  the  transformation  of  the  Analytic  System.  The  OSD 
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Analytic  Agenda  was  created  to  achieve  this  purpose.  It  recognizes  the 
changing  nature  of  warfare  brought  about  by  new  technologies,  new  missions 
and  an  enemy  that  is  intelligent,  unpredictable  and  continuously  varying  its 
patterns. 


Analytic  Agenda 

J  00, 000  Foot  Level 

DoD-wide  agreement  to  make  major,  joint  analysis  efforts  more 
effective,  efficient,  relevant 


Our  current  modeling  and  simulation  suites  were  evolved  from  Cold  War  models 
and  have  served  us  well.  As  we  have  continued  to  improve  legacy  models  we 
have  also  invested  a  considerable  effort  to  develop  newer,  more  robust  models 
to  support  the  defense  analytic  process.  However,  the  majority  of  modeling 
activity  since  the  Cold  War  ended  has  continued  to  center  on  the  analysis  that 
supports  traditional  warfighting  as  we  moved  from  the  two  simultaneous  Major 
Theater  Warfare  strategies  of  the  1990s  to  the  sequential  Major  Combat 
Operations  strategy  of  this  decade.  As  we  attempt  to  seek  ways  to  develop 
techniques  to  analyze  the  non-traditiona!  areas,  we  have  found  that  our  familiar 
existing  suite  of  tools  is  not  well  suited  to  examine  the  capabilities  of  the  Joint 
force  against  these  new  challenges.  This  leads  to  the  need  for  a  workshop  to 
facilitate  cross-community  identification  and  sharing  of  tools,  new  techniques  and 
methodologies  (such  as  agent  based  modeling)  and  new  developments.  The 
workshop  is  expected  to  provide  immediate  benefit  to  modeling  developments 
and  analytic  support  to  provide  insights  to  answer  post  2005  QDR  questions 
within  the  Department  and  in  support  of  near-term  component  analytic  plans  and 
future  Analyses  of  Alternatives. 


2.  Sponsor  interest 

Navy  and  Air  Force  have  agreed  to  be  co-proponents  of  this  workshop.  The  idea 
of  a  workshop  to  expand  on  the  Navy’s  internal  World  Class  Modeling  Workshop 
in  June  2005  was  originally  discussed  at  the  MORS  Sponsors’  luncheon  at  the 
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73rd  MORSS,  The  meeting  was  again  discussed  during  a  World  Class  Modeling 
presentation  at  the  21  July  2005  Joint  Analytic  Data  Management  Steering 
Committee  (JADM  SC)  and  all  of  the  remaining  MORS  Sponsors  indicated 
interest  and  support.  Following  a  meeting  between  the  co-sponsors  and  OSD 
representatives,  the  concept  was  briefed  at  the  JADM  SC  on  25  August. 

3.  Purpose,  Goals  and  Objectives 

The  purpose  of  the  special  meeting  is  to  match  stakeholder  analytic  gaps  and 
deficiencies  with  potential  methodology  and  modeling  solutions  in  support  of  new 
developments  in  defense  analytic  requirements  and  capabilities.  Specifically,  the 
workshop  is  intended  to; 

•  Illuminate  user  Non-traditional  modeling  and  simulation  gaps  and  deficiencies 
in  the  context  of  the  defense  security  challenges  and  emerging  changes  in 
the  defense  strategy 

•  Identify  areas  of  user  commonality 

•  Capitalize  on  Joint/Service/Agency  toolsets  and  ongoing  model  development 
for  the  larger  community 

•  Develop  partnerships  to  share  future  M&S  development 

The  workshop  will  help  to  achieve  the  following  longer-term  goals  and  end-states 
for  the  DoD  community  and  workshop  stakeholders: 

•  Improve  DoD  analysis  tools  used  for  requirements  determination, 
effectiveness  evaluation  and  program  planning 

•  Develop  the  ability  to  model  emerging  security  challenges  and  new 
operational  concepts:  Irregular,  Catastrophic,  Disruptive  challenges, 
FORCEnet,  Effects-Based  Ops,  Distributed  Ops  concepts 

•  Improve  traceability  of  data  through  modeling  hierarchy:  Engineering  to 
Engagement  to  Mission  to  Campaign 

•  Advance  state  of  the  art  in  M&S  technology,  practices,  standards,  and 
architectures 

•  Inform  DoD  analytic  M&S  community  of  important  related  activities 

The  following  objectives  will  ensure  a  successful  workshop: 

•  For  Each  Working  Group: 

o  Identify  key  stakeholder  decision-makers  [pre-workshop  objective] 
o  Examine  analysis  gaps  and  deficiencies  needs  from  a  stakeholder 
perspective  [pre-workshop  objective] 
o  Determine  commonalities  between  stakeholders 

o  Examine  common  gaps  and  deficiencies  and  identify  potential  solutions  in 
terms  of  methodologies,  legacy  M&S,  modifications  of  legacy  M&S  or  new 
M&S  developments  from  a  broader,  enterprise-wide  perspective 
o  Explore  potential  partnerships  for  new  M&S  developments 
o  Rank  the  M&S  needs  within  each  working  group  to  provide  an  important 
advisory  input  for  the  Analytic  M&S  Master  plan 

4.  Approach 
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The  Analysis  for  Non-Traditional  Security  Challenges  Workshop  will  be  organized 
into  five  working  groups  and  one  synthesis  group  which  will  address  gaps  and 
deficiencies  in  assessing  the  three  non-traditional  challenges  (irregular, 
catastrophic  and  disruptive),  deterrence  and  decision-making/collaboration 
methods.  The  workshop  is  predicated  on  analysis  stakeholders  having 
completed  their  gap  analysis  prior  to  the  workshop  so  they  can  provide  their 
needs  as  an  input.  This  requires  stakeholders  to  be  identified  3-4  months  prior  to 
the  conference  to  enable  them  to  have  time  to  complete  the  prerequisite 
analysis.  Each  breakout  working  session  will  then  identify  and  compile 
commonalities  and  seek  solutions.  A  solution  could  be  in  terms  of  an  existing  or 
new  methodology  such  as  an  experiment  or  exercise  lesson  learned,  a  legacy  or 
improvement  to  a  legacy  tool  or  a  new  M&.S  development.  Overlapping/common 
gaps  between  stakeholders  will  be  identified  as  potential  for  partnering.  This 
could  lead  to  a  common  study,  or  co-sponsored  new  development. 

Concurrent  with  the  workshop  planning  will  be  an  IDA  led  effort  to  develop  the 
DoD  Analysis  M&S  Master  Plan.  To  support  this  effort,  an  initial  workshop  is 
being  planned  in  the  November  timeframe  to  identify  M&S  needs  and  M&S 
communities  of  interest.  It  is  anticipated  this  workshop  will  not  only  help 
stakeholders  begin  the  process  for  identifying  gaps  and  deficiencies,  but  may 
also  identify  additional  stakeholders  for  the  MORS  workshop.  Subsequent  M&S 
Master  Plan  workshops  should  also  help  to  identify  and  the  range  of  tools  and 
even  planned  developments  that  can  feed  this  workshop. 

Two  MORS  workshops  have  been  held  this  fall  that  will  be  leveraged  for  the  Non- 
Traditional  Workshop.  The  workshop  on  “Agent  Based  Models  and  Other 
Analytic  Tools  in  Support  of  Stability  Operations,  was  held  at  the  SAIC 
Conference  Center  in  McLean  in  late  October  and  “Homeland  Security  - 
Homeland  Defense  Decision  Support"  was  held  in  November  at  JHU/APL. 
Products  from  these  workshops  will  be  filtered  to  provide  inputs  as  appropriate  to 
this  workshop. 

WG-1  Irreaular/GWOT —  Irregular  challenges  to  U.S.  security  interests  are 
characterized  by  methods  such  as  terrorism  and  insurgency  with  the  intent  of 
eroding  U.S.  influence,  patience,  and  political  will.  Irregular  opponents  often  take 
a  long-term  approach,  attempting  to  impose  prohibitive  human  material,  financial 
and  political  costs  on  the  U.S.  to  compel  strategic  retreat  from  a  key  region  or 
course  of  action.  Our  experience  with  the  war  on  terrorism  points  to  the  need  to 
reorient  our  capabilities  to  contend  with  this  irregular  challenge  more  effectively. 
This  working  group  will  identify  methods,  tools  and  models  that  will  focus  on  gaps 
and  deficiencies  for  Irregular/GWOT  areas  such  as: 

•  Counter-insurgency 

•  Counter-terrorism 

•  Distributed  Operations 

•  Small  Unit  Operations 

•  Riverine  Operations 

•  Foreign  Internal  Defense 
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•  Critical  infrastructure  Protection 

•  Maritime  Domain  Awareness 

•  Terrorists  Networks 

•  Partnering/Shaping 

WG-2  Disruptive  —  In  rare  instances,  revolutionary  technology  and  associated 
military  innovation  can  fundamentally  alter  long-established  concepts  of  warfare. 
Some  potential  adversaries  are  seeking  such  disruptive  capabilities  to  exploit 
U.S.  vulnerabilities  and  offset  the  current  advantages  of  the  U.S.  and  its  partners. 
Disruptive  breakthroughs  in  areas  such  as  bio-technology,  cyber-operations, 
space,  or  directed-energy  weapons  could  seriously  endanger  our  security. 

Identify  maritime,  land  and  air  transportation  tools,  models,  methods  and  metrics. 
This  working  group  will  identify  methods,  tools  and  models  that  will  focus  on  gaps 
and  deficiencies  for  Disruptive  areas  such  as: 

•  Red  and  Blue  Swarming  Operations 

•  Bio-technology/Nanotechnology 

•  Stability  Operations 

•  CNA  Defense 

WG-3  Catastrophic  —  In  the  face  of  American  dominance  in  traditional  forms  of 
warfare,  some  hostile  forces  are  seeking  to  acquire  catastrophic  capabilities, 
particularly  weapons  of  mass  destruction  (WMD).  Porous  international  borders, 
weak  international  controls,  and  easy  access  to  information-related  technologies 
facilitate  these  efforts.  Particularly  troublesome  is  the  nexus  of  transnational 
terrorists,  proliferation,  and  problem  states  that  possess  or  seek  WMD, 
increasing  the  risk  of  WMD  attack  against  the  United  States.  Proliferation  of 
WMD  technology  and  expertise  makes  contending  with  catastrophic  challenges 
an  urgent  priority.  Even  a  single  catastrophic  attack  against  the  United  States 
would  be  unacceptable.  We  will  place  greater  emphasis  on  those  capabilities 
that  enable  us  to  dissuade  others  from  acquiring  catastrophic  capabilities,  to 
deter  their  use  and,  when  necessary  to  defeat  them  before  they  can  be 
employed.  This  working  group  will  identify  methods,  tools  and  models  that  will 
focus  on  gaps  and  deficiencies  for  catastrophic  areas  such  as: 

•  WMD  Elimination 

•  Effect  of  Nuciear/Biological  attack 

•  Consequence  Management/Foreign  Consequence  Management 

•  WMD  Detection 

WG-4  Deterrence  —  The  Strategic  Deterrence  Joint  Operating  Concept  (SD 
JOC)  defines  strategic  deterrence  as  the  prevention  of  adversary  aggression  or 
coercion  threatening  vital  interests  of  the  United  States  and/or  our  national 
survival.  The  central  idea  of  the  SD  JOC  is  to  exercise  decisive  influence  over  a 
potential  adversary’s  strategic  deterrence  Center  of  Gravity:  the  decision-making 
calculus  of  key  adversary  decision-makers.  Joint  military  operations  and 
activities  contribute  to  the  '‘end"  of  strategic  deterrence  by  affecting  the 
adversary's  decision  calculus  elements: 
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•  Denying  Benefits 

•  Imposing  Costs 

•  Inducing  Adversary  Restraint 

Military  means  required  to  affect  the  decision  calculus  will  vary  significantly 
depending  on  the  adversary  but  fall  into  two  categories:  those  that  directly  and 
decisively  influence  and  adversary’s  decision  calculus  and  those  that  enable 
such  decisive  influence. 

Enabling  “means"  include: 

•  Global  Situational  Awareness 

•  Command  and  Control 

•  Overseas  presence 

•  Allied  Military  Cooperation  and  Integration 

Direct  "means"  include: 

•  Force  Projection 

•  Nuclear  Strike  Capabilities 

•  Active  and  Passive  Defenses 

•  Global  Strike 

•  Strategic  Deterrence  Information  Operations 

•  Inducement  Operations 

•  Space  Control 

The  SD  JOC  describes  how  each  of  these  "means"  contributes  to  the  “ways”  of 
achieving  the  strategic  deterrence  "end"  and  identifies  how  the  joint  force 
attributes  identified  in  Joint  Operations  Concepts  impacts  these  capabilities. 

This  working  group  will  identify  methods,  tools  and  models  that  will  focus  on  gaps 
and  deficiencies  for  assessing  the  "means”  to  achieve  the  “ways"  for  the  strategic 
deterrence  end.  New  deterrence  concepts  (global,  transnational  and  regional} 
proposed  by  the  QDR  may  also  be  considered. 

WG-5  Decision  Making  and  Integration  —  Effective  decision-making  requires 
new  methods  and  tools  to  incorporate  Non-Traditional  Challenges.  Existing 
methods  of  collaboration  will  need  to  be  reevaluated  to  consider  the  effects  of 
incorporating  the  new  challenges  across  all  levels  of  classification.  New 
developments  in  adaptive  planning  and  course  of  action  analysis  must  account 
for  these  new  areas  as  well. 

Synthesis  Group —  By  definition,  these  working  groups  are  not  mutually 
exclusive.  The  inherent  overlaps  between  the  working  groups  provides  synthesis 
points  for  integrating  the  conclusions  from  each  as  well  as  reducing  the 
probability  that  major  ideas  will  "fall  through  the  cracks"  between  the  workshop 
topics.  The  synthesis  group  will  examine  cross-group  commonalities  and  insights 
and  develop  the  overall  workshops  recommendations. 
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5.  Workshop  Planning  and  WG  Chairs 

Navy/Air  Force  Co-Proponents: 

Dr  Jacqueline  Henningsen,  FS,  Air  Force 
Mr  Trip  Barber,  Navy 

Workshop  Sponsor/Service  POCs: 

Mr  Royce  Reiss  (AFSAA)  Co-Sponsor  POC 
Ms  Sunny  Conwell  (N81 )  Co-Sponsor  POC 
Ms  Renee  Carlucci  (MCCDC) 

Mr  John  Robertson  (Army  G-3) 

Program  Chairs: 

Mr  Lee  Dick  (N8 1  /Axiom  Corp)  iee.dick@naw.mil.  703-693-8890 
Co-Chair:  Dr  Jim  Harris  (AFSAA)  James.Harris@Dentaaon.af.mil 
703-588-8294 

Workshop  Advisors:  Mr  Jim  Bexfield,  FS,  OSD 

Dr  David  Markowitz  (CAA) 

Technical  Advisor:  Trena  Lilly,  JHU/APL 

WG1-  Irregular/GWOT 

Chair:  Dr  Ed  McGrady,  CNA 

Co-Chair:  Mr  Mike  Ottenberg,  OSD-PAE/ATT 

Recorder:  TBD 

WG  2  -  Disruptive 

Chair:  Dr  Mike  Bailey,  MCCDC 
Co-Chair:  Mr  Steve  Farley,  N81/ONR 
Recorder:  TBD 

WG  3  -  Catastrophic 

Chair:  Dr  Al  Sweetser,  OSD  (PA&E) 

Co-Chair:  Mr  Walt  Zimmers,  DTRA 
Recorder:  TBD 

WG  4  -  Deterrence 

Chair:  LTC  Steve  Riese,  STRATCOM 
Co-Chair:  Mr  Mark  Reid,  MITRE 
Recorder:  TBD 

WG  5  —  Decision  Making  and  Integration 

Chair:  Prof  Greg  Parnell,  FS,  USMA 
Co-Chair:  Mr  Steve  Richter,  JHU/APL 
Recorder:  TBD 
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WG  6  -  Synthesis 

Chair:  Mr  Royce  Reiss,  AFSAA 
Co-Chair:  Ms  Sunny  Conwell,  N81 

6.  Attendance 

Facility  limitations  will  require  attendance  to  be  capped  at  about  250  people.  The 
goal  is  to  seek  the  right  blend  of  stakeholders  and  industry/government 
developers.  Quotas  will  be  assigned  to  stakeholders  and  industry  organization 
limits  may  be  required  to  ensure  diverse  attendance.  Invitations  are  expected  to 
be  extended  to  foreign  nationals  and  may  include  non-TTCP  countries.  Target 
attendance  will  be  about  35  per  working  group  excluding  the  Synthesis  Group 
depending  on  breakout  room  accommodations.  The  WAL  has  been  reserved  for 
WG-1  and  will  accommodate  up  to  80  with  50  seats  reserved  for  using 
groupware. 

7.  Deliverables 

Several  products  will  be  generated  from  the  workshop: 

•  An  Executive  Summary  will  be  presented  to  the  sponsors  at  the  next  JADM 
Steering  Committee  meeting  subsequent  to  the  workshop.  The  summary  will 
address  the  workshop  objectives,  findings,  conclusions  and 
recommendations.  This  will  be  in  the  form  of  a  report  and  a  scripted  briefing 
that  lists  gaps  and  shortfalls  between  the  communities  and  opportunities  for 
cooperation.  This  will  include  identification  of  current  tools  used  by  the 
communities  and  potential  areas  for  tool  and  data  sharing,  as  well  as  current 
repositories  of  data  and  information. 

•  A  proceedings  document  containing  summaries  of  all  sessions  in  the  form  of 
annotated  copies  of  plenary  breakout  session  briefings. 

•  A  detailed  report  of  gaps  and  linkages  will  be  provided  as  feedback  to  the 
Analysis  M&S  Master  plan.  This  will  likely  be  in  spreadsheet  form. 

•  An  article  summarizing  the  meeting  and  its  findings  will  be  produced  and 
submitted  to  PHALANX  in  time  for  the  next  deadline  after  the  meeting. 

•  A  general  session  presentation  will  be  made  at  the  74th  MORSS. 
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8.  Milestone  Table 


31  Aug  05 
8  Sep  05 


Initial  Organizing  Committee  meeting 
EC  approves  TOR 

Identify  Working  Group  Chairs  and  Stakeholders 
Pre-Planning  workshop  products  due 
Workshop 

Post  Workshop  brief  to  sponsors  at  JADM  SC 

Draft  article  for  PHALANX 

Workshop  Brief  at  MORSS  Special  Session 


Draft  TOR 


Oct  05 
Oct  05 


31  Jan  06 
21-23  Feb  06 


Mar  06 
Mar  06 
Jun  06 


9.  Planning  and  Organizing  Committee 

General  Chair:  Mr  Lee  Dick  N81 

General  Co-Chair:  Dr  Jim  Harris,  AFSAA 

Advisors:  Mr  Jim  Bexfield,  FS,  OSD(PA&E) 

Dr  David  Markowitz  (CAA) 

Synthesis  Chair:  Mr  Roy  Reiss,  AFSAA 

Synthesis  Co-Chair:  Ms  Sunny  Conweli,  N81 
Synthesis  WG:  Dr  Tom  Allen  FS,  IDA 


Mr  Greg  Keethler,  LMCO 
John  Robertson,  Army  G-3 
Renee  Carlucci,  MCCDC 
Jim  Stevens,  OSD  PA&E 
Bob  Orlov,  Joint  Staff  J8 
Dr  Roy  Rice,  FS,  TBE 


Site  Coordinator/Poster  Session  Coordinator:  Trena  Lilly,  JHU/APL 

Administrative  Coordinators:  Brian  Engler,  Executive  Vice-President,  MORS 
and  Natalie  Kelly,  Vice-President  for  Administration,  MORS 
MORS  Bulldog:  Annie  Patenaude 

Sponsor/Service  Reps: 

Air  Force:  Roy  Reiss,  AFSAA 

Army:  COL  Hoa  Generazio,  ODUSA(OR) 

Navy:  Herb  Cupo,  N81 

Marine  Corps:  Col  Greg  Reuss,  MCCDC  S&A 
Joint  Staff:  Bob  Orlov,  Joint  Staff  (J8) 

OSD:  Jim  Bexfield,  FS,  OSD(PA&E) 

10.  Administrative 

Name:  Analysis  for  Non-Traditional  Security  Challenges:  Methods  and  Tools 
Dates:  21-23  February  2006 
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Location:  The  Johns  Hopkins  University/Applied  Physics  Laboratory,  Laurel,  MD 
Fees:  US  Federal  Government  $310  and  $545  for  all  others;  Plenary  (1  day 
only)  Government  $160;  Non-Government  $280 
Attendance:  250  people,  by  invitation 

Classification:  The  Workshop  will  be  unclassified.  Presentations  with  FOUO 
and/or  limited  distribution  markings  will  not  be  distributed  electronically  or  made 
available  as  a  printed  copy  to  the  general  audience.  Distribution  of  these 
materials  will  be  determined  by  the  originator.  Post  workshop  products  such  as 
the  PHALANX  article,  executive  outbrief,  and  75th  MORSS  presentation  will  be 
cleared  for  public  release  and  may  be  posted  on  the  website. 

11.  Agenda 

Process  for  Day  1 


Morning: 

0700-0800 

0800-0810 

0810-0820 

0820-0900 

0900-0915 

0915-0940 

0940-1010 

1010-1030 

1030-1050 

1050-1110 

1110-1130 

Afternoon 

1130-1300 

1300-1430 

1430-1445 

1445-1700 

1700 

Check  in  and  Registration 

MORS  President  Welcome 

Sponsor  Welcome/Keynote  Intro 

Navy  Sponsor  Welcome  and  Keynote  Speaker  Intro 

Break 

Analysis  M&S  Master  Plan  Update  [IDA] 

QDR  Debrief  -  Gaps  and  Shortfalls  [OSD(PA&E)/J8] 

Navy  WCM  Initiatives 

AFSAA  M&S  Initiatives 

Center  for  Army  Analysis  M&S  Initiatives 

MCCDC  M&S  Initiatives 

Working  lunch  in  breakout  rooms/WG  intros/way  ahead 
COCOM  Panel  (J-8  chair) 

Break 

Breakout  sessions  in  WGs 

Mixer 

Process  for  Dav  2 

0800-1600 

1600-1700 

Breakout  sessions 

10  min  plenary  updates  by  WG  chairs 

Process  for  Dav  3 

0800-0940 

0940-1000 

1000-1200 

1300-1415 

1415-1430 

1430-1545 

1545-1600 

1600-1630 

Breakout  sessions  continue 

Break 

Prep  WG  Session  outbriefs 

WG  session  out  briefs  (25  min  per  WG  +  Synthesis  Group) 
Break 

Continue  session  out  briefs 

Break 

Sponsor  concluding  remarks 
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